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About This Guide 


Introduction 


Using data to make decisions about students is challenging. For many years, educators have been asked 
to use their professional judgment, without readily available data, to determine student needs. The 
amount of data available to inform these decisions has increased exponentially; still, the sheer amount 
and access to these data continue to make using data to identify student needs a challenge. The 
Massachusetts Department of Elementary and Secondary Education (ESE) is focused on supporting 
educators in the use of data; to this end, ESE developed an Early Warning Indicator System (EWIS) that 
rolls many student data variables into a single indicator that provides educators with information about 
which students are at low, moderate, or high risk of missing key academic milestones as they enter 
school each fall. 


Annually, the Massachusetts EWIS provides information about students’ risk levels for every student in 
grades 1-12 who attended a public school in Massachusetts in the year prior. EWIS is based on several 
years of research that examined student data and identified the combination of data points that most 
accurately predict the likelihood that a student will reach or fall short of achieving a key educational 
milestone. The risk levels of students are provided to Massachusetts’ districts and schools in the late 
summer each year. EWIS data provides educators with critical information about whether students are 
on track for the next important academic milestone. Identifying students early on in the school year 
allows educators to take action to make schoolwide changes to minimize risk, provide appropriate 
interventions and supports to identified students when they first enter school in the fall, and monitor 
student progress throughout the school year to ensure that students stay on track. 


Purpose of the Guide 


The purpose of this guide is to provide information on how to use early warning data, including the 
Massachusetts Early Warning Indicator System (EWIS), to identify, diagnose, support and monitor 
students in grades 1-12. It offers educators an overview of EWIS and how to effectively use these data in 
conjunction with local data by following a six-step implementation cycle. 


Because no two districts, schools, or students are alike, we purposely developed this guide as a flexible 
tool to help educators customize their use of EWIS to fit their individual needs, interests, and resources. 
We hope that this guide will help you with the important work you do every day in your classrooms, 
schools, and districts. 


This guide is divided into the following four main sections: 


Section 1. What are Early Warning Data? This section defines early warning data and provides 
background information on the Massachusetts EWIS, including how it was developed, how it works, 
and instructions for accessing EWIS data. Districts/schools new to EWIS will find this section 
particularly helpful for understanding EWIS, and getting ready to use early warning data for school 
improvement and student support purposes. 


Section 2. The Six-Step Early Warning Implementation Process. This section introduces schools and 
districts to a six-step early warning implementation process and is organized into subsections for 
each step in the cycle. Each subsection briefly describes the step, anticipated outcomes, and the 
tasks and processes schools and districts can use to implement the step. “Notes from the Field” 
anecdotes derived from Massachusetts districts and schools that have piloted an early warning 
implementation process, are included to offer readers promising approaches, lessons learned and 
concrete examples for implementation from varying district and school contexts. “Build off Your 
Foundation” includes prompts that encourage schools and districts to build upon the systems and 
practices that are already in place when using early warning data. Each step includes a set of 
guiding questions to help you move through the implementation process and outlines some of the 
ways in which districts can support early warning data use. Finally, each step concludes with some 
tools that may be helpful for implementation of the step. 


Section 3. Additional Resources. This section provides information and links to additional 
Massachusetts and other resources that may be beneficial. 


Section 4. Appendixes. This section provides additional information to support implementation and 
understanding of early warning data. 


Section 1. What are Early Warning Data? 


Across the United States, districts and schools are using 
early warning data to predict the likelihood that a student 
will meet or miss key academic milestones, such as 
graduating from high school. Research has found that Is your district or school new (or relatively 
readily available student data sources such as new) to the use of early warning data? This 


attendance, behavioral records, and course failures, can section of the Guide will be especially 
helpful in preparing you and your team to 
understand EWIS and be ready to use early 
warning data in your school or district. 


Implementation Tip: 
Learn More About EWIS 


be used as early warning indicators to identify students 
who are at academic risk. For example, data on student 
attendance, when combined with a research-based 
threshold such as missing 10% of school days, can be 
used to “flag” students who are at risk of not graduating from high school. Early warning data also 


includes locally determined indicators and diagnostic data, such as local benchmark assessments and 
demographic information, that schools and districts use to identify and diagnose the needs of students, 
provide interventions and supports, and track student progress throughout the year. Districts and 
schools also use early warning data to inform schoolwide strategies to address student needs, and to 
examine long-term, school-level patterns to address systemic issues that may impede a school’s or a 
student’s ability to meet academic milestones. Although a key aspect of using early warning data is the 
identification of students, it is essential that such identification does not result in another label for 
students but rather in actionable information that leads to support for students in need. In 
Massachusetts these data are integral to the implementation of the Massachusetts Tiered System of 


Support (MTSS): http://www.doe.mass.edu/sfss/ 
About the Massachusetts EWIS 


The Massachusetts EWIS is an early warning system that identifies students who are at risk of missing 
key academic milestones in grades 1-12. EWIS student risk levels are computed annually at the end of 
each summer as students begin the school year. The Massachusetts EWIS was developed in direct 
response to educators’ requests for information that enables them to identify students who may be at 
risk at earlier grade levels and throughout high school. Working with American Institutes for Research 
(AIR), ESE intentionally designed EWIS to rely solely on data available through existing statewide 
collections of multiple data sources. Using a rigorous statistical method, researchers created a valid 
EWIS model for each grade level, from elementary school through high school.” The EWIS development 
process was several years in the making. The EWIS risk models reflect national research and promising 
practices. Extensive testing of possible data sources, and statistical modeling using longitudinal 
Massachusetts data, were used to create three student risk levels that comprise the current EWIS. (For 
more information on the development of the EWIS model and risk levels, see Appendix A or 


http://www.doe.mass.edu/ccr/ewi for the technical documentation.) 


EWIS Academic Milestones 


Massachusetts EWIS student risk levels are predictive of relevant academic milestones. A student’s EWIS 
risk level predicts the likelihood that the student will or will not meet a particular academic milestone. 


In EWIS, student risk is organized by four grade level groupings. These age groups are as follows: early 
elementary, late elementary, middle grades, and high school. For each age group, a student’s EWIS risk 
level predicts the likelihood that the student will or will not meet an academic milestone identified for 
that particular age group. The academic milestones that have been identified for each age group are 
developmentally appropriate, based on available state data, important to the success of Massachusetts’ 
students, and meaningful and actionable for adult educators who work with the students in each grade 
grouping (Exhibit 1). For example, the early elementary age group encompasses grades 1—3 and assesses 
risk based on the academic milestone of achieving a score of Meets or Exceeds Expectations on the 
grade 3 English Language Arts (ELA) state assessment, or MCAS. Scoring Meets or Exceeds Expectations 
on the grade 3 ELA MCAS is a proxy for reading by the end of grade 3 and is a developmentally 
appropriate benchmark for children in the early grades. In Fall 2016, EWIS expanded to include three 
postsecondary milestones (College Enrollment, Academic Readiness and College Persistence). 


Exhibit 1. Massachusetts Age Groups, Grade Levels, and Academic Milestones 


| Grade | Academic milestone 
Age Groups : 
Levels (Expected Student Outcome for Each Age Group) 
Early 1-3 Reading by the end of grade 3 
elementary Meeting or Exceeding Expectations on the grade 3 ELA MCAS 


Middle school ready 


Meeting or Exceeding Expectations on the grade 6 ELA and 
N mathematics MCAS 


«| Middle High school ready 
7-9 f ‘ 
grades Passing grades in all grade 9 courses 


Late 
elementary 


Postsecondary 


High school graduation 
Completing all local and state graduation requirements in four 
years. 
Hiehschoal 10-12 Enrolling in postsecondary education 
Academic Readiness 
Enrolling in credit-bearing courses without developmental 
education 
et eee | Persistence 
et eee | in a second year of postsecondary education 


The academic milestones for early elementary, late elementary, and the middle grades also are positively 


associated with high school graduation (the high school academic milestone). For example, research points to 
the importance of reading by the end of grade 3 as an important milestone for students to be on track for 


ultimately graduating from high school.? Students who do not score proficient’ on the third grade state 
assessment are 3.7 times less likely to graduate from high school. 


EWIS Student Risk Levels 


There are three risk levels in EWIS: low, moderate, and high. These risk levels relate to a student’s predicted 
likelihood for reaching his or her age group’s defined academic milestone. A risk level quickly provides 
information to educators about the likelihood that an individual student will or will not achieve an academic 
milestone. In other words, the risk level indicates whether a student is currently on track to reach the 
upcoming academic milestone. For example, a student who is identified as low risk is predicted as likely to 
meet the academic milestone. 


The EWIS risk levels are determined by using data from the previous school year. A risk level is assigned 
to every Massachusetts public school student for whom there are state-level data from the prior year. 
The risk levels are determined on an individual student basis and are not based on a student’s relative 
likelihood of reaching an academic milestone when compared with other students. As a result, there are 
no set amounts of students at each risk level. For example, it is possible to have all students in the low- 
risk category. Exhibit 2 overviews the EWIS risk levels. 


Exhibit 2. EWIS Student Risk Levels 


Likely to reach the upcoming academic milestone. Approximately 90 percent of 
Low Risk students who are at low risk will meet this academic milestone within each age 

group. 

Moderately at risk for not reaching the upcoming academic milestone. 
Moderate Risk Approximately 60 percent of students at moderate risk meet this academic 

milestone within each age group. 


At risk for not reaching the upcoming academic milestone. Approximately 


High Risk 25 percent of students at high risk meet this academic milestone within each 


age group. 


For more information about EWIS, including the data included to determine risk and the background of 
the development of the EWIS, see Appendix A. 


Accessing EWIS Data 


EWIS data for Massachusetts students are available to districts and schools through Edwin Analytics”. ESE 
developed a range of EWIS reports to facilitate educators’ use of this information. EWIS data also can be 
exported to Excel and combined with other student data sources. EWIS data are made available in late 


1 This data point was calculated using the legacy MCAS tests which had proficiency as key achievement level. 


? Edwin is a comprehensive system that includes Edwin Analytics, available to all MA districts. 
EEE 
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summer of each school year and provide information on each student who attended a Massachusetts 
public school in the prior year. Districts and schools can access EWIS data through Edwin Analytics, which is 
found by using the ESE website security portal, available through the upper right side of the ESE website: 
http://www.doe.mass.edu/. The data also can be directly accessed at https://gateway.edu.state.ma.us/. 


Each district’s directory administrator provide staff access to Edwin Analytics. The list of directory 


administrators can be found at http://www.doe.mass.edu/infoservices/data/diradmin/list.aspx. For more 


information about using Edwin and the EWIS data and reports, 


see Appendix B or http://www.doe.mass.edu/edwin/analytics/. Implementation Tip: 
Accessing EWIS Data 


= Access EWIS reports in Edwin 
Analytics: 
https://gateway.edu.state.ma.us, 


About Monitoring Indicators 


Contact your district’s directory 
administrator to request access: 


EWIS student risk levels offer districts and schools information 


on students based on the prior year’s data. Monitoring 
http://www.doe.mass.edu/infoservices 


indi rs, which readily available local data, can 
ae ENMGe teacily a /data/diradmin/list.aspx 


complement the use of EWIS data. After receiving the EWIS risk 
levels for students at the beginning of the year, monitoring 


indicators are a means by which educators can track student progress and flag students as in need 
during the course of a school year. For example, a grade 2 student who is identified by EWIS as being at 
high risk of not scoring Meets or Exceeds Expectations on the grade 3 ELA State Assessment may be 
provided with reading support at the beginning of the school year. The team may then use attendance, 
disciplinary data, and reading benchmark data as monitoring indicators several times during the year to 
make sure this student stays on track. In addition to using monitoring indicators to observe the progress 
of individual students, districts and schools may use monitoring indicators to identify trends in student 
needs and make recommendations for schoolwide improvements such as changes in schedules, 
curriculum, or instructional approaches. Tool 3 in Step 1 overviews potential monitoring indicators and 
indicates whether they are research based or commonly used but need to be locally validated. Teams 
will select monitoring indicators as part of their work in Step 1: Get Organized. 


Early Warning Implementation Key Terms 


Having a common language for talking about, understanding, and using early warning data to support 
student needs may help your team use early warning data more effectively. Throughout the guide there 
are several key terms that are used, which are described in Box 1. 


Box 1. Early Warning Implementation Key Terms 


Early warning data: Early warning data is a general term that refers to data used to predict, 
diagnose, and monitor student risk for not meeting academic milestones. Early warning data 
refers to EWIS student risk levels and other data found in the EWIS reports, as well as local data 
sources which include monitoring indicators and diagnostic data. 


Early warning implementation: The multi-step process that districts and school can follow to 
effectively use early warning data to identify at-risk students, explore underlying causes for risk, 
provide interventions and supports, monitor student progress, and make schoolwide 
improvements to lower risk for all students. This multi-step process is a key foundation for the 
Massachusetts Tiered System of Support (MTSS). 


Massachusetts Early Warning Indicator System (EWIS): The early warning system created by 
the Massachusetts Department of Elementary and Secondary Education that identifies students 
who are at risk of missing key academic milestones in grades 1-12. The four key academic 
milestones in K-12 are reading by the end of third grade, middle school ready, passing all grade 
9 courses, and high school graduation. The three postsecondary milestones included for 
students in high school are college enrollment, academic readiness, and college persistence. The 
EWIS provides student risk levels at the beginning of each school year. Student-level and 
aggregate EWIS information is available through several reports in Edwin Analytics. 


EWIS Student Risk Level: Each student who attended a Massachusetts public school in the year 
prior is assigned a level of risk of low, moderate, or high. A student’s risk level indicates how 
likely it is that the student will miss the key academic milestone for their grade level, such as 
passing all their 9°" grade courses or graduating from high school. EWIS student risk levels are 
based on statistical formulas that are applied to multiple sources of readily available data from 
the prior year. Students who are assigned a student risk level of high are predicted to be most 
off-track for reaching the next key academic milestone. Conversely, students who are assigned a 
risk level of low, are predicted to be on-track for reaching the next key academic milestone. 


Local Data: Schools and districts can complement their use of EWIS with local data sources. 
There are two types of local data sources: monitoring indicators and diagnostic data. 


= Monitoring indicators. Monitoring indicators rely on data that are available during the 
school year. Data sources such as attendance, academic performance, and behavior have 
been identified in research as strong predictors of student risk for missing key academic 
milestones. Monitoring indicators complement the use of EWIS student risk levels. After 
receiving EWIS student risk levels at the start of the school year, Massachusetts’ districts 
can use monitoring indicators to flag students who are at risk and track student progress at 
multiple times throughout a single school year. Monitoring indicators can either be 
research-based or locally determined. 


Diagnostic Data. Additional data sources, such as demographic data, benchmark data, and 
state-level student data available through Edwin, is used by schools and districts in 
combination with EWIS student risk levels and monitoring indicators. These supplemental 
data sources can help schools and districts better understand underlying causes of student 
risk, and track student progress in interventions. 


Section 2: The Six-Step Early Warning Implementation Process 


To maximize the value of early warning data, educators can use early warning data within the context of 


a multistep, data-driven implementation process. This allows educators to use early warning data to 


match interventions and supports to student needs, monitor student progress to ensure that students 


stay on track and are responding to interventions, and strengthen schools along the way. 


The Early Warning Implementation Theory of Action 


Exhibit 3 illustrates the early warning implementation theory of action and how it is intended to support 


student success. The implementation process is supported by three components (as shown in the left 


column of Exhibit 3): (1) EWIS, monitoring indicators, and diagnostic data; (2) the Early Warning 
Implementation Guide and other training, resources and supports; and (3) the district and/or school 


team or staff dedicated to the use of these data. Exhibit 3 illustrates how these components contribute 


to improved student outcomes. 


Exhibit 3. Theory of Action for Early Warning Implementation 
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It is expected that the combined use of these components will (1) improve educators’ use of data to 


identify students who are at risk for not meeting academic milestones, (2) improve educators’ ability to 
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efficiently and effectively match interventions and supports to meet student needs and make broader 
schoolwide improvements to reduce risk for all students, and (3) improve educators’ ability to monitor 
student progress over time. The arrow at the bottom of the illustration represents an expected cycle, 
where progress through the pathway from A to C yields continued improvements in these intermediate 
outcomes. The intermediate outcomes, in turn, are expected to promote critical student outcomes, 
particularly students’ likelihood of getting back on track by meeting EWIS benchmarks (D) and, 
ultimately improved student success in meeting academic milestones (E). 


Getting Ready for Early Warning Implementation: Team Exercise 


Before learning more about the six-step early warning implementation process, staff may want to 
become more familiar with early warning data, and consider how its use will best fit with what they are 
already doing in their district or school. The following two activities may help you get ready to use early 
warning data effectively as you work to support students and drive improvements. 


Box 2. Getting Ready for Early Warning Implementation: Team Exercise 


Explore EWIS: Take a spin through your current EWIS data and reports in Edwin Analytics. Start 
by running the EWIS District View Report (EW301). What did you notice about the extent of risk 
for your district or school? Which school or grade had the highest proportion of students at risk? 
The lowest? Did you see anything in your EWIS data that was surprising? What new questions 
did you have after reviewing your EWIS data? What other uses can you foresee for your EWIS 
data? Who might want to use it-why and when? 


Where does early warning implementation fit? Consider the various ways that your district or 
school is already using student data? How is data used to determine Tier 2 and Tier 3 supports 
and interventions and/or to evaluate their effectiveness? Which individual staff members, 
committees, or teams use data to identify and support students? Which committees or teams 
use (or could use) data to inform decisions about school improvement efforts? Considering your 
existing data uses, committees, teams, and improvement initiatives—where and how might 
early warning implementation fit? Would early warning data add value or focus to these existing 
efforts? Are you better off convening a new and separate group to specifically focus on using 
early warning data? What are your next steps for determining who, when, and how you will use 
early warning data in your school or district? 


Overview of the Six-Step Early Warning Implementation Cycle 


This guide outlines a six-step EWIS implementation process adapted from AIR’s National High School 
Center publications on early warning system implementation.* The process draws on research on data 
use in districts and schools? in the context of implementing an early warning system, such as EWIS, and 


S) 


is similar to a cycle of inquiry, such as the one outlined in the Massachusetts District Data Team Toolkit 


(http://www.doe.mass.edu/accountability/toolkit/ ). 


In this guide, the six steps are presented in a sequence, but they are intended to be cyclical. Exhibit 4 
illustrates the cycle. At the core of this data-driven, decision-making process, the steps focus users on 
using EWIS data in combination with local data, including monitoring indicators (e.g., attendance, 
behavior, course performance) to identify which students are showing signs of risk of missing key 
academic milestones and diagnostic data which helps teams know how to take action. Teams use data 
to identify, diagnose, assign, provide, and monitor interventions and supports for individuals and groups of 
students as well as make schoolwide improvements to lower the risk for all students. The six steps guide 
users through a straightforward process that supports informed decisions based on these data and other 
information. 


Exhibit 4. Six-Step Early Warning Implementation Process 


STEP 1: 
Get Organized 
Summer/ 
Beginning of 
School Year 


STEP 2: 
Review EWIS Data 


STEP 6: 
Refine the 
Process 
STEP 3: 
Explore Underlying 
Causes 
End of School STEP 5: 
Year/ Summer Monitor Students 
and Interventions 
Repeat Throughout 
School Year 


STEP 4: 
Take Action 


Adapted with permission from the Early Warning Intervention Monitoring System™ implementation guides for the middle grades and high school, 
developed by the National High School Center. (See Endnote 1.) Copyright © 2013 American Institutes for Research. All rights reserved. 


10 


The remainder of Section 2 of this guide outlines the tasks and the processes for implementing each 
step in the cycle. The guide is intended to be a flexible tool to help educators customize their use of 
early warning data to fit their school’s individual needs, interests, and resources. No two schools are 
alike, so early warning implementation may vary across districts and even among schools within a 
district. For example, some schools or districts may find that they have a high percentage of students 
who are identified as being at risk. For these schools and districts, their focus may be primarily on 
schoolwide changes in addition to interventions and supports for individual students and groups of 
students identified as being in need. Another district may find that they have very few students who are 
identified as being at risk. This district may focus its efforts primarily on providing support and 
interventions to meet the needs of individual students. 


Implementation Timeframe 


The early warning implementation process is carried out during the course of a school year and is 
aligned with the academic calendar. Specific steps are undertaken at defined periods of the year, many 
in a recurring or a continuous manner. As the timeline shown in Exhibit 5 illustrates, teams begin to 
organize their work prior to the start of the school year (Step 1). Typically, teams will implement Steps 2, 
3, and 4 during the first six weeks of the school year. After student interventions and supports have 
been assigned (Step 4), teams periodically review monitoring indicators to ensure that support 
strategies and interventions are responsive to student needs and use monitoring indicators to ensure 
that all students remain on-track (Step 5). As shown, it is recommended that teams monitor students at 
the end of each grading period. In the spring, teams use monitoring indicators data to inform the 
planning of summer supports for students. Finally, at the conclusion of the school year, teams refine 
their approach to using early warning data (Step 6). 


Implementing the Six-Step Process 


The following subsections offer detailed, step-by-step information to guide implementation efforts 
through the six-step implementation cycle. Each step includes the following: 


= What you need to implement the step 


= Description of the step 


Notes from the Field 


= Anticipated outcomes 


= Notes from the Field, anecdotes and concrete examples for early warning 


implementation in Massachusetts 


= Build on your Foundation, ideas for how your team might build upon 
current systems and practices 


= School and district roles 


=" Guiding questions 


= Tools, such as step implementation checklists, worksheets, and action planning tools 
SEES eae 
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Exhibit 5. Early Warning Implementation Timeline 
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Step 1: Get Organized 


What You Need for Step 1 


= Interest and commitment from appropriate administrators and staff from the school and/or the 
district who are able to meet regularly as a team to use early warning data to identify and support 
students in need 


= Sufficient time to meet regularly throughout the school year (at a minimum, a monthly meeting 
lasting one to two hours) 


= Access to EWIS data and reports within Edwin Analytics 
Description of Step 1 


There are three critical parts to getting organized. 


1. Convening a team or group (existing or new) that will be responsible for reviewing early warning 
data and making decisions about supporting students in need. 


2. Devising the plan for the year, including your teams’ goals, meeting times and frequency, data 
import schedule, and how you will communicate and coordinate with others. 


3. Getting ready to use early warning data including accessing EWIS student risk levels in Edwin 
Analytics and identifying which data your team will use to monitor students over the course of 
the school year. 


A. Convening the Team(s) 


A diverse, well-informed team is essential to the effective use 


of early warning data for identifying and supporting students. Build Off Your Foundation 
The team desiesiatse as responsible 10 alc EWIS eats ane seit cl acne meer ec alend 
other early warning data, such as monitoring flags, to identify and support students in need? 
and support students may be established as a new team. = How do youcurrently monitor 
Alternatively, a school or district may choose to build upon the student progress? 

work of an existing team (e.g., the school data team, the = How can you integrate the use of 
school improvement team, the System for Student Success EWIS and other early warning 


data into the work of your current 


team, and/or the student support team). Regardless of 


teams/committees? 
whether the team is newly formed or integrated with an = Are there ways youcan merge 
existing team, the team should include a broad array of staff EWIS data and other early 


within the school and/or district and should be prepared to warming data with your current 
. -_ naeer Meee : data systems? 

dedicate sufficient meeting time to reviewing, interpreting, 

and acting on student early warning data. To help you get 


organized, refer to Tool 4 in Step 1. 


Anticipated Outcomes for Convening the Team 


= The identification of an existing team(s), or the 


convening of a new team(s) at the school and/or ; 
district level Notes from the Field 


= The selection of team members who have a diverse set 


fk ledge and skills and are committed to the use 
One To have the K-12 lens on our 


Students and such a variety of 
students different people talking about 
student needs really broke down 


of early warning data for identifying and supporting 


= Clarification of roles and responsibilities of the team, 


silos. Nothing happens ina 


including the identification of one or more individuals bubble — so to get all the 
to serve as chair and co-chair, logistical support, and a different perspectives focused 
data specialist (who is responsible for importing and on one issue was huge. 


managing early warning data) 
Structuring the Team 


Early warning implementation requires the engagement of 


stakeholders at multiple levels. Although much of the 

action of identifying underlying causes and supporting students in need occurs within school-level 
teams, the district has an important role to play. Teams may vary in structure, number, and 
composition, depending on the size of the district. Large districts with many schools may have one 
school-based team at each school as well as a district-level team with both school and district 
representation. Smaller districts may have one school-based team in each school with both school and 
district representation. For example, in one school, the school-based team may be composed of a 
content area teacher (e.g., reading or mathematics), a school counselor, a technical/data person, and a 
principal or an assistant principal. The school team may meet monthly, and the district administrator 
may attend every other month. Such a scenario streamlines communications between the district and 
the school regarding implementation and allows team members to identify implementation challenges 
that are best resolved or addressed by the district. 


The following briefly describes the role of both the school and the district in the implementation process 
associated with using early warning data. 


School Role. The primary role of each school is to use EWIS and other early warning data to identify and 
support students within the school, and to help these students meet desired academic milestones. 
Within each school, the team will draw on available information, coordinate and collaborate with 
existing committees and groups, and bring together additional educators and specialists to explore 
schoolwide changes. The team, in consultation with other stakeholders, including students and family 
members, can match individual students to appropriate interventions and supports and develop 
programs and supports to meet the needs of groups of students. Each school also may want to be in 
regular communication with its feeder school(s) to facilitate transitions and assist district administrators 
aL 


14 


in coordinating efforts across the district. The intensity and the frequency of activities and 


communication associated with the use of early warning data will vary by school but are likely to be 
greatest in the fall. 


District Role. The role of the district is to support efforts within schools to use and respond to early 
warning data, facilitate communication and coordination across schools (particularly for school 
transitions), and identify and address system-wide concerns. For example, a district might observe that 
many students coming from specific elementary schools are more likely to be identified as being at high 
or moderate risk after the first year of middle school. The district may want to examine the underlying 
causes for this pattern. Perhaps one feeder elementary school is not preparing its students for middle 
school in the same way that other elementary schools are. Or perhaps the particular needs of these 
students are addressed in elementary school but are not sufficiently addressed in the first year of 
middle school. In addition, district administrators play a key role in establishing a mission, goals, and a 
direction for the early warning work, and are well-placed to communicate the importance of EWIS and 
other early warning data. The district also can play a critical role in ensuring that school teams have 
access to data, resources, training, and other supports necessary for effective implementation. 


Team Composition 


Strong teams include a diverse, broad, and layered membership that represents a range of skills and 
expertise (e.g., leadership, technical expertise, instructional expertise, social/emotional, and mental 
health expertise.) It is suggested that the team(s) consist of personnel who have both the authority to 
make decisions about interventions and supports, staff, and students, and familiarity with a diverse 
array of students, including students with disabilities and students who are English language learners. In 
addition, it is critical to have at least one individual with the skills to access and analyze EWIS data (in 
Edwin Analytics) and the ability to import and analyze additional early warning data from other student 
data. Over the long term, membership on the team will likely evolve and shift. Although it is good to 
rotate and engage more staff in the process over time, it is helpful if some individuals continue to serve 
on the team across multiple years to ensure continuity and consistency. 


Implementation Tip: 
Potential Team Members 


School-based team (may include district, school and community reps): 
School principal or assistant principal 
Representatives from feeder schools (i.e., elementary, middle) 
District office representative 


Special education teachers 


English language learner (ELL) instructors 


School counselors 


School psychologist or social worker 


Content area teachers 


Data specialist (or someone proficient in using the school’s student data systems) 


Afterschool director 
District-based team (may include district, school and community reps): 
Superintendent or assistant superintendent 
Data specialists 
Testing/assessment/accountability administrators 
Curriculum and instruction representatives 
Pupil personnel/ student support administrators 
Special education administrators 
ELL administrators 
Principal/administrator or another administrator from each school 
School counselors and/or social workers 
Classroom teacher representatives 


Others (e.g., members of other district and school teams and committees) 


Team Roles and Responsibilities 


Whether the school or district team is integrating early warning implementation work into an existing 
committee, team, or group, or forming a brand new team, it will be important for the team to function 
effectively, efficiently, and professionally. One of the most important initial steps is to determine the roles 
and the responsibilities of the team members. It is suggested that the team designate a chair and a co- 
chair to provide leadership, facilitate meetings, and serve as the primary liaisons to stakeholders outside 
the group. Many teams also may want to designate an individual to provide logistical and administrative 
support to the team. Finally, most teams dedicated to using early warning data will want to designate at 
least one individual, either on the team or in support of the team, who will act as the primary data 
specialist. This should be someone with proficiency in using and linking student data systems with other 
databases, and someone who has sufficient time available to manage the importing of student data, and 
production of early warning data reports. 


Guiding Questions for Convening the Team 


= Will the team be a newly established group or integrated into an existing committee, team, or 
group? 


= Will there be a separate district-level team or only district representation on school teams? 
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= What types of staff need to be represented on the team (e.g., district administrators, counselors, 
teachers, and/or data specialist)? 


= What types of knowledge and expertise do team members need to have (e.g., knowledge of student 
needs, diverse student populations, awareness of family and cultural factors, data analysis 
strategies, existing supports, and/or dropout prevention programs)? 


= How might the composition of the team differ across school levels within a district (e.g., should 
membership be different at the elementary versus middle versus high school levels)? 


= Who will serve in key roles on the team (e.g., chair, administrator/manager, note taker, data 
specialist)? 


B. Devising the Plan 


Effective teams articulate clear goals and procedures for effective team functioning and meetings, 
establish regular meeting times, and devise a plan for communicating and coordinating with others. 


Anticipated Outcomes for Devising the Plan 


= Clarity and consensus regarding how the team will conduct its work (e.g. clear, written goals and 
objectives, and plans for how the group will conduct effective meetings) 


= Schedule of regular meeting times to review, interpret, and act on early warning data throughout 
the school year. 


= Clear plan for how the team will communicate and coordinate with others outside the team, 
including how the team will protect the confidentiality of student data. 


Team Goals and Functioning 


The team(s) dedicated to the use of student early warning data will meet to review student data, 
explore the underlying causes of risk, and align students in need with appropriate interventions and 
supports. The team also may want to explore group or schoolwide changes that may meet the needs of 
a broader range of students. Throughout the year, continuous monitoring of students who display risk 
flags will improve the team’s ability to assign appropriate interventions or supports and will allow 
midcourse corrections if a particular student or a group of students does not respond to an intervention 
or support. The team can facilitate its functioning and enhance effective communication throughout the 
school by doing the following: 


= Setting clear written goals and objectives for the work of the team. This will be particularly 
important in the first year of implementation. Many teams have found it helpful to narrow the focus 
and expectations for their work during the first year to allow them to refine team functioning and 
effective use of early warning data before attempting schoolwide implementation. 


= Establishing rules and procedures for protecting the confidentiality of student data. Teams will 
need to determine who will have access to sensitive student data and include written instructions 
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and provisions for protecting confidentiality on reports 
and other documents containing sensitive information. 


Communicating with individuals and groups outside 
the team. Teams may want to share information with a 
range of other individuals and groups in their efforts to 
support students, including other school and district 
committees and teams, teachers and specialists, and 
social service agencies, and community based 
organizations such as afterschool providers, who are 
already, or could be providing services to students 
identified as at risk. For example, teams may want to 
communicate regularly with school and district 
leadership about students in need, intervention or 


Notes from the Field 


Our EWIS school team 
shared EWIS data with 
guidance and school 
adjustment counselors. These 
counselors were able to use 


EWIS data when developing 
Individualized Student 
Support Plans as part of their 
Response to Intervention 


(RTI) work. 


support effectiveness, and team-identified needs for 
support. Teams also may want to provide teachers with 
regular updates about students in their classes who are 
displaying risk flags, as well as information about 


supports available to teachers in working with these 
students. Teams may want to collaborate with afterschool providers to align in and out of school 
support for at-risk students. Teams may want to establish procedures and rules regarding how, 
when, and with whom they will communicate information associated with their activities and 
findings and how they will protect the confidentiality of student information being shared. 


= Communicating and collaborating with students and families. In some cases, the team may want to 
communicate with students and families about their concerns, or convey these concerns through a 
teacher. Collaborating with students or families may help to ensure that students are adequately 
supported and that all interested parties have a chance to help the student get back on track. Of 
critical note, the team should share concerns about students with sensitivity; sharing specific data 
about EWIS risk levels is not advised. It is important to ensure that any shared information is used to 
prompt action and support, not give labels that carry stigma. 


For example, a student with an EWIS student risk level of “high” may have consistently poor 
attendance in the new school year. Rather than sharing with the student and the family that the 
student is “high risk,” a more productive and specific approach is to inform the student and the 
family that the poor attendance is likely contributing to being off track for meeting an important 
academic milestone (e.g., passing all grade 9 courses). This approach allows for a discussion for 
possible reasons for poor attendance that may be addressed. 


Team Meetings 


Strong teams set aside adequate time to accomplish their work and establish a consistent meeting 
schedule, space, and set of procedures. The following are a few strategies to support effective and 


efficient team meetings. 
ez 
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Meet regularly to monitor student progress. The team will want to meet regularly (within the first 
30 days of school and at least at the end of each grading period) to monitor progress as it strives to 
improve the educational outcomes for students. Many teams find that they need to meet more 
frequently in the fall in order to implement the initial steps in the implementation cycle. This portion 
of the cycle ensures that students identified as at-risk at the start of school can be assigned to 
appropriate interventions. It will be important to schedule meetings around the availability of 
student data, allowing sufficient time for these data to be entered and analyzed prior to the team’s 
meeting date. 


Conduct well-organized and documented team meetings. Clear agendas are important to ensure 
that each meeting has goals and a focus. At least some agenda items will likely be routine, such as a 
review of student data, actions taken for individual students or groups of students, a review of 
previous meeting action items (ongoing or completed), and communication with staff and leadership. 
Meeting minutes or notes can help teams to document decisions and organize any additional 
information that has been gathered and/or analyzed by the team, and help them keep track of action 
items and assigned tasks. 


Establish procedures for making team decisions and assigning work. Team members have many 
responsibilities, and limited time is often an impediment to reaching team goals. Having procedures 
in place for making team decisions fairly and efficiently will help meetings to go more smoothly. In 
addition, many teams find that having clear roles, 
responsibilities, and procedures for delegating tasks 
enhances the team’s effectiveness. 


7 Implement the steps outlined in the 

Notes from the Field implementation cycle. Once organized, the work of the 
team will focus on implementing the steps outlined in 
the implementation cycle (i.e., generating early warning 


We realized that it doesn't need data reports, reviewing and interpreting early warning 
to be another meeting! The 
EWIS work can be incorporated og: 
trying to create a new team of knowledgeable about this cycle and receive training on 
professionals who are already how to implement key steps associated with the use of 
taxed and pulled in a variety of 
directions on a daily basis. 


data, assigning interventions and supports, and 


student early warning data. 


Guiding Questions for Devising the Plan 


" What are the team’s priorities, principles, and 
expectations regarding how the team will conduct its 


work (i.e., what is the group’s process)? 
How frequently and on what specific dates should the team meet? 


What does the school-level team hope to accomplish? What does the district-level team hope to 
accomplish? 
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= How will the team members ensure they have sufficient time and resources to meet and conduct 
their work outside EWIS meetings? 


= What types of technical support and/or professional development are needed to train and support 
the team(s) at the school and district levels)? 


= How and with whom will the team communicate about the team’s mission, activities, and progress? 


C. Getting Ready to Use Early Warning Data 


An early warning system uses readily available student data to identify students who are at risk of not 
meeting desired academic milestones. To launch your early warning implementation process you will 
need to identify and access the student data the team will use to identify, support, and monitor the 

progress of students in need of support. 


Anticipated Outcomes for Getting Ready to Use Early Warning Data 


= Access EWIS data and reports 


At least one member of each team will have proficiency in: accessing EWIS data (through Edwin 
Analytics); using the various reporting options for examining student risk information; and 
exporting EWIS student-level data (EW601 and/or EW602) to Excel for further analysis. 


All members of each team will understand the four academic milestones in EWIS, be familiar 
with the types of data underlying the risk levels, and have the ability to understand the various 


EWIS reports. 


= Preparing for the monitoring of student risk 


The team will have identified which student data 
sources will be used to monitor student risk 
throughout the year. 


The team will have established thresholds that will 
be used to “flag” students as being at risk based on 
these data sources. 


The team will have selected (or set up) the data 
system or tool that will be used for monitoring 
student risk flags. 


At least one member of the team, or someone 
supporting the team, will have proficiency in 
updating monitoring indicator data and generating 
reports using the monitoring indicators database 
system or tool. 


Notes from the Field 


Prior to EWIS, we relied only on 
guidance counselors and 
teachers to identify students 
having academic difficulty and 
bring them to the Instructional 
Support Team. The EWIS data 
has inspired us to develop our 
own academic warning system, 
which expanded upon the EWIS 
tool by incorporating additional 
data points. This system allows 
us to systematically combine 
data on all students, including 
professional perspectives. 
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Accessing EWIS Data and Reports 


EWIS data reports in Edwin Analytics provide information on students’ risk-level status at the beginning 
of each school. The team will need to access these data to understand the extent of student risk at the 
beginning of the school year. EWIS data and reports provide information to districts and schools on 
students’ risk levels based on data from the prior year. ESE applies risk formulas to existing data for all 
Massachusetts public school districts, resulting in an assigned level of risk for every student. Within 
Edwin Analytics, districts and schools can view their EWIS data through several report options: 


= District View (EW301) and Postsecondary District View (EW303) 

= Graphical View (EW302) 

= Risk Level Indicator Analysis (EW317) and Postsecondary Risk Indicator Analysis (EW3019) 
= Subgroup Analysis (EW318) 


= Student List (EW601) and Postsecondary Student List (EW602) 


Within each option, districts and schools can filter the data in numerous ways (e.g., select specific schools, 
grade levels, and/or student subgroups) to gain varying perspectives on their data. These reports allow 
districts and schools to understand the scope, the depth, and the nature of student risk levels. EWIS 
data are accessed from Edwin Analytics, which is available through ESE’s Security Portal. The Security 
Portal can be found in the upper-right side of the ESE website (http://www.doe.mass.edu/) or directly at 
https://gateway.edu.state.ma.us/. For more information about accessing Edwin Analytics and the EWIS 


data and reports, please see Appendix B or http://www.doe.mass.edu/edwin/analytics/. 
Exporting EWIS Data 


All reports in Edwin Analytics may be exported to PDF or Excel formats, which may be helpful for sharing 
and further analysis. Some districts may want to expand their capacity to examine EWIS data beyond the 
reporting options offered from Edwin Analytics and therefore may choose to export their student-level 
EWIS data (Edwin Analytics report EW601 and EW602) into Excel. After the EWIS data are exported, 
districts and schools can combine these student-level data (which includes all of the data points that 
were used to assign a student to a particular risk level) with other sources of district and/or school data 
and/or generate a wider range of data reports. 


Preparing to Monitor Students 


One of the keys for using early warning data is the frequent monitoring of student progress using readily 
available data. Monitoring indicators can be used during the school year to flag students in need, and 
track progress. Data such as attendance, academic performance, and behavior have been identified in 
research as strong monitoring indicators. Monitoring indicators can complement the use of EWIS data 

by enabling teams to flag students who become or continue to be at risk during the school year. 
Monitoring indicators can also be used to track students’ responses to interventions or supports at 
multiple times throughout a single school year and thereby assist teams in making decisions regarding 
_— ee 
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interventions and supports. Diagnostic data, another local data source, such as benchmark assessment 
data, student focus groups, and staff professional opinions, can be used in conjunction with research- 
based monitoring indicators. Diagnostic data is particularly helpful in exploring the underlying causes for 
student risk, and tracking student progress in interventions. 


Getting Ready to Monitor Students and Interventions Implementation Tip: 
Go Slow 

During Step 1, teams prepare to use early warning data, 

including monitoring indicator data before or during the Proceed slowly when generating data 

early part of the school year. These steps need to be 

completed prior to monitoring students and 


reports to avoid an overload of 
information. Remember, data are useful 
only when team members can understand 
interventions: and make use of the information provided. 


= Select which student data sources the team will use 


to monitor student risk throughout the year. 


= Establish thresholds to “flag” students as being at risk using these data sources (e.g., students who 
miss 10 percent or more of school days). 


= Establish a system or tool your team will use to monitor students. 


= If using a data system or tool, make sure that at least one member of the team has proficiency in 
importing, updating, and generating reports using the database system or tool that contains student 
data. 


= If using a data system or tool, establish a data import and review schedule for analyzing data and 
reports with up-to-date student monitoring indicators data throughout the school year. 


= Verify that all members of the team understand the data sources for monitoring student progress and 
the various monitoring indicator reports. 


Selecting Monitoring Indicators 


Teams can select from one or more monitoring indicators. Decisions about monitoring indicators should 
be grounded in research or local validation of data. A monitoring indicator draws from readily available 
data that can be observed at many points during the school year. As shown in Tool 3 below, monitoring 
indicators are data sources with identified thresholds, defined in research or locally validated whereby 
users can observe whether a student is flagged because he or she met or exceeded this predetermined 
threshold. For example, if the data source is attendance and the threshold for flagging students is 
missing 10 percent of school days, then any student who misses 10 percent or more of school days 
would be flagged as at-risk. The frequency with which flags are examined is dependent on the schedule 
set by the team. 


Selection Considerations. Before, or at the start of the school year, districts and schools must select the 
specific monitoring indicators they will use to track student progress throughout the year. The guiding 


principles for selecting monitoring indicators are as follows: 
SSS SSS SSSSSSSSSSSSSqqqqNQNmNINqNVIyNy)SN_—eNSoOOHoOH!!,_SSS—BN_ TI 
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(1) Indicators are research based or locally validated. 


(2) The indicator is based on readily available data that can be updated and accessed several times 
throughout the school year. 


(3) More than one, but no more than four indicators per grade 
level, are used by the team to monitor student risk Implementation Tip: 
throughout the year. Remember, monitoring indicators are Be Discerning in Your Selection of 
used to identify students who are displaying symptoms of Monitoring Indicators 
need, not to diagnose need. 


Each grade level should only use 2-4 
monitoring indicators since these 

Tool 3, some districts and schools may also want (or need) to use indicators are used just to flag students 
other local data sources to flag and track student progress using throughout the school year (not to 
diagnose their needs). If you are new to 
using monitoring indicators, you may want 
to start with only 1 or 2 monitoring 

or a school uses a commercially available benchmark assessment indicators per grade level. Using 

tool, to determine students’ academic progress in reading, then attendance as a monitoring indicator is 
often a good place to start. Attendance can 
be a powerful monitoring indicator and 
easily accessible. 


After considering the list of possible monitoring indicators in 


data that is more closely aligned with the needs and practices of 
their district, schools, and/or students. For example, if a district 


this information may be a source of data that can be used to 
identify at-risk students and track student progress during the 


year. Some schools and districts may want to select data sources 
and establish performance thresholds more closely associated with particular supports to monitor 
student progress in these interventions. 


Data Systems for Monitoring Students and Interventions 


It is recommended that the team identify or establish a 
Notes from the Field system before, or early on in the school year, to manage 
the data that will be used to track monitoring indicators. 
The team has various options for managing these data. For 
For monitoring students, we instance, districts and schools may wish to use their own 


decided to use attendance, student information systems for monitoring students. 
grades, “Orleans-Hannah” and 


: Districts and schools may choose to use an existing system 
“Galileo” data, as well as our 


for monitoring students, such as AIR’s National High School 


reading inventory from the 
beginning of the year. Center EWS High School Tool or EWS Middle School Tool® 


which is available for free, or other commercially available 


monitoring indicator tools. Note that in these cases there 
must be a person who is dedicated to importing student 
data into the tool several times over the course of the 
school year. In either case, the team must ensure that it 
has access to needed student data throughout the year, 


and has a schedule for data access. 
a ——————— 
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Guiding Questions for Getting Ready to Use Early Warning Data 


= EWIS Data and Reports 


Do all team members understand the academic milestones for the age groups on which the 
team is focused? Do all team members have a general understanding of how the risk levels were 
determined? 


Is there someone on the team who knows how to access EWIS information and generate a range 
of reports? Who will be responsible for generating the EWIS reports? 


Does the team have an interest in generating more reports or combining EWIS data with other 
student data? Does the team have someone who knows how to export EWIS data and merge 
these data with other data systems? 


With whom will EWIS data be shared? How will student confidentiality be protected, as required 
by district policies? Who will provide reports to other stakeholders? 


= Monitoring Indicators 


Which data sources make the most sense for our school to use to monitor its students? Will our 
team use the research-based indicators from Tool 3? Should we use our own locally validated 
indicators? 


What thresholds will the team set to determine when students should be “flagged” as at risk? 
Are these selected data sources and thresholds research-based or can they be validated locally? 


Does the team want to use any diagnostic data sources to track student progress in assigned 
interventions, such as available benchmark assessment data? 


What data systems or procedures does the school already have in place for tracking student 
progress? Does the team want to use its own data system or import data into a commercially 
available monitoring indicator tool? How frequently will the team examine risk flags based on 
these data sources? 


Who will be responsible for obtaining the data to monitor students? Who will generate data 
reports showing students flagged as at-risk, for the team? 


With whom will these data be shared? How will student confidentiality be protected, as 
required by district policies? Who will provide reports to other stakeholders? 


District Role in Step 1 


The role of the district and/or the district team is to identify system-wide concerns and develop and 


recommend districtwide changes that address such concerns. District administrators also play a key role 


in establishing a mission, goals, and direction for the early warning work and communicating the 


importance of early warning data, including EWIS, within and across schools. District administrators can 


participate in the work of school teams, provide professional development, and support and oversee 


school-level efforts. To maximize the impact of the district role, we recommend the following: 
SaaS EES SSS ESS SSS 
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= Engage in school-level meetings or routinely communicate with teams to increase attention to 


efforts and signify the importance of using early warning data. For example, on one team, a district 
administrator might participate in a team meeting at designated times during the school year. This 
allows the team to share information about students in the school and identify challenges and needs 
based on their use of early warning data. The district administrator benefits by having the 
opportunity to hear about challenges facing the team and consider ways in which the district may 
support this work. For example, the district administrator might assign a data specialist to provide the 
team with information in a particular, more user-friendly manner, or connect the team with a school 
that has had success in supporting students. 


= Provide professional development to team members on using EWIS or other student early data. This 
training can enhance the work of the team and decrease variation in the quality of the team’s work. 
For example, a district may offer professional development to new school-level team members at 
the end of each school year or during the summer to prepare them for the work ahead. 


= Arrange for school-level team members to have release time, coverage, or other supports to enable 
them to participate in meetings and related work. 


= Ensure that school-level teams have been prepared for and have access to student-level data (e.g., 
knowledge of confidentiality procedures/guidelines, passwords, and instructions for accessing). 


= Monitor school efforts to use early warning data throughout the school year and over the course of 
multiple school years to ensure that schools are improving outcomes for students. This allows the 
identification of promising practices and areas of need in the district as a whole. Districts may 
monitor and track early data use and implementation by participating in meetings or requesting 
presentations on school-level work. This gives the district the opportunity to identify common needs 
across schools and develop districtwide strategies for allocating resources or sharing practices. 


= Ensure that teams have access to EWIS data and reports in Edwin Analytics. 
= Ensure that at least one or more members have been trained in using Edwin Analytics. 


= Provide technical support (either directly or through a third party) to team members in accessing and 
modifying the current data system to track specified indicators or in learning how to use existing 
monitoring indicator tools (if preferred). 


= Work with school-level teams on selecting the student data sources and thresholds that will be used to 
monitor student progress that allow districts to have the desired consistency in data collection across 
schools. 


If there is a district-level team in place, this team may meet less frequently than school-based teams 
(e.g., two to four times per year) to discuss: persistent problems and challenges; resources and 
strategies for supporting students; and systemic, organizational, and policy changes that may be 
needed. School-level representatives can help the district team develop new districtwide strategies for 
students in need (e.g., new policies, schedules, partnerships, curriculum, or training for teachers and 
students). 


a) 
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Tool 1 — Step 1: Checklist 


The Team 


Determine team structure (e.g., new team or existing team? School level, 
district level, combined, or both?) 

Determine team composition (who will be part of the team?) 

Determine team roles and responsibilities (e.g., leader, data designee, 
logistics) 


The Plan 


Determine rules and expectations related to the team’s group process 
(decision-making, conflict resolution, and ground rules) 

Communicate with the district to clarify goals and expectations (district and 
school) 

Set team goals, objectives, timeline and tasks 


Construct a team communication plan 
Set team meeting schedule (timing, frequency, and length) 


The Data 
O Confirm EWIS data access 
O Select locally meaningful monitoring indicators for observing changes in 


student risk over the course of the school year 
O Confirm that monitoring indicator data sources are accessible and 
readily available 


O Assign responsibility for providing the team with monitoring indicator data 


and reports in preparation for team meetings 


O Identify other local data sources to be used in diagnosing student needs and 


tracking progress in interventions 
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Tool 2 — Step 1: Where Are You Now? 


Directions: Form small groups of 3-4 people. Share with each other where you are now with your use of 
data and your current approaches for identifying and supporting students in need. You may discuss all 
the questions, or focus on one or two in more depth. 


Discussion Questions: 


1. How do you currently identify and support students in need? 


What types of data do you use to identify students in need? 
Do you use EWIS data to identify students in need? If yes, how do you use these data? 
Do you use other student data? How do you use these data? 


Who is responsible for identifying students in need and ensuring that support is provided to 
these students? Is this the role of one individual, several individuals, or a team? 


2. How do you currently monitor student progress? 


Do you examine data to identify students in need during the school year? 
How frequently do you monitor the progress of students who are identified as in-need? 


What database are you currently using to organize these data? 


3. How can you integrate the use of early warning data, including EWIS, with your current 
practices? 


Which aspects of your current approaches for identifying and supporting students in need 
do you feel are adequate or good? 


Are there changes or modifications you would like to make to your strategies for identifying 
and supporting students in need? What are they? 


How do you think your use of early warning data and the early warning implementation 
process can be integrated with or strengthen your current practices? 
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Tool 3 — Step 1: Potential Monitoring Indicators by Grade Level 


Directions: The table below contains possible monitoring indicators to use throughout the school year for each grade level. Each grade level 
includes attendance, behavior, and academic performance potential monitoring indicators flags students that are on or off track. Some of these 
monitoring indicators are more strongly supported by research (marked in the “Research Supported” column), while others are commonly used 
but need to be locally determined and validated to establish an appropriate threshold. In the “Consider Using?” column, check of which 
monitoring indicators your school or district could potentially use and note the data source and possible threshold for the monitoring indicator. 
If you are new to using monitoring indicators, you may want to start with only 1 or 2 monitoring indicators per grade level. 


Academic Potential Receareh recall Consider | Notes — existing data source, 
milestone | Grade | Monitoring Threshold u Using? possible threshold, etc. 
Wdicstor Supported Determined 


Early Elementary 


Behavior Locally defined behavior grade, office 
discipline referrals 

Academic Locally defined performance levels on 

performance Benchmark assessments 


2 


Academic Locally defined performance levels on 

performance Benchmark assessments 

Attendance Locally defined 

Behavior Locally defined behavior grade, office 
discipline referrals 

Academic Locally defined performance levels on 

performance Benchmark assessments 


Additional notes about possible monitoring indicators for grades 1, 2, and 3: 
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Behavior Locally defined behavior grade, office 
discipline referrals 


on grade 3 ELA State Assessment 
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Academic Potential Consider | Notes — existing data source, 


Locall 
milestone Monitoring Threshold Berearch eeally Using? possible threshold, etc. 
Indicator Supported Determined 


Late Elementary —— | 
Attendance 20% or more school days missed 


Behavior Locally defined behavior grade, office 
discipline referrals 

Academic Locally defined performance levels on 

performance Benchmark assessments 


ae Se a ee 
Ee ee ee 
Behavior Locally defined behavior grade, office X 
discipline referrals 
Course Failing grade mathematics and/or ELA X 
performance 
[ae A (O] 


20% or more school days missed 
5 


Academic Locally defined performance levels on 
performance Benchmark assessments 


= 20% or more school days missed 


Meets or Exceeds Expectations on grade 6 
ELA and mathematics State Assessment 


Behavior Locally defined behavior grade, office 
discipline referrals 

Course Failing grade in mathematics and/or ELA 

performance 

Academic Locally defined performance levels on 

performance Benchmark assessments 


Additional notes about possible monitoring indicators for grades 4, 5, and 6: 
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Academic Potential Consider | Notes — existing data source, 
Research Locally 


Supported Determined 


Middle School —— | 
20% or more school days missed 
7 


milestone Monitoring Threshold 
Indicator 


Using? possible threshold, etc. 


Pe 

fea 

patos [wemmenmmnermes | 

performance 

Schavior [Suspensions offs dcpinerefevas | | x» | +S 

iAtendance _[20%ormeresshooldays | xT SSSSC*dSC“‘<‘d/CSC“‘(CSC*’ 

Eine femmes 

performance 

schavior [Suspensions offeedscpineefevas | |» (| —*+|—S—~CS 
ee a ee 
re 
ee 
a ee 


; Attendance 10% or more school days missed 


Passing grades in 
all grade 9 courses 


Course Failing one or more grading period 

performance 

Credits Enough credits for promotion to next 
grade 


Behavior Suspensions, office discipline referrals 


Additional notes about possible monitoring indicators for grades 7, 8, and 9: 
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Indicator threshold, etc. 


10% or more school days missed 


Course Failing one or more grading period 
performance 


Behavior Suspensions, office discipline referrals 


Academic Potential Threshold Research Locally Consider Notes — existing data 
milestone Monitoring Supported Validated Using? source, possible 


Behavior Suspensions, office discipline referrals 


11 Attendance Locally defined percentage of school 
days missed 

12 Attendance Locally defined percentage of school 
days missed 


ae | 
ae Ce a a 
Course Locally defined number of course X 
performance failures 
a (a 
aS es es ee 


High school graduation 


Course Locally defined number of course 
performance failures 


Behavior Suspensions, office discipline referrals 


Additional notes about possible monitoring indicators for grades 10, 11, and 12: 
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Tool 4 — Step 1: Early Warning Implementation Action Planning Tool 


Directions: The following template is designed to support your early warning implementation efforts for Step 1. Teams can identify the key 
objective or task, and then identify the resources available, actions needed responsible parties, and when the task needs to be completed. 


A. Team Structure, Roles, and Responsibilities 


Key Tasks: 


Determine whether team will be newly formed, or integrated into an existing committee or team. 

Determine team structure (school level, district level, or combined school and district level membership). 

Determine team composition (who will be part of the team). 

Determine team roles and responsibilities (e.g., team facilitator, data management designee, interventions specialist, team 
logistics). 

Determine meeting schedule (timing, frequency, and length). 

Clarify team’s group process (decision making, conflict resolution, and ground rules). 

Communicate with district leadership to clarify goals and expectations (district and school). 

Set team goals, objectives, tasks and timelines. 

Construct a team communication plan. 


B. Get Ready to Use Early Warning Data 


Key Tasks — EWIS Data and Reports: 


¢« Designate a person to manage EWIS data and 
generate EWIS reports. 

¢ Obtain permission to access EWIS data 

¢ Ensure that district and/or school student data are 
up-to-date. 

¢« Determine how student confidentiality will be 
protected. 

¢ Ensure that team members understand EWIS (risk 
levels, academic milestones) and how to interpret 
data. 


Key Tasks — Monitoring Indicators: 


* Designate a person to manage monitoring indicator data 
and generate reports. 

* Determine final list of monitoring indicators. 

e Establish thresholds for locally determined monitoring 
indicators. 

¢« Determine how student confidentiality will be protected. 

¢« Set schedule for updating monitoring indicator data and 
generating reports. 

e« Ensure that team members understand how to interpret 
monitoring indicator data and reports. 
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Step 2: Review EWIS Data 


What You Need for Step 2 
= Access to Edwin Analytics for generating EWIS reports 
= Exploring your Early Warning Data Worksheet (see Tool 7 in Step 2) 


= Sufficient time for team members to analyze existing data and reports and generate additional 
reports 


Description of Step 2 


In Step 2, the team explores EWIS data to answer important questions about students in need and 
understand patterns in student engagement and academic 
performance within a school and/or district. The data 

reports in Edwin Analytics are tools for seeking answers to 
the team’s questions. Build Off Your Foundation 


= What questions or challenges at 


When teams first start looking at the EWIS data, it can be al ari aera 


overwhelming. It takes time to develop a strategy to are you already focused on? 
review the information. During the first year of Consider exploring these 
. : : : questions in your EWIS data 
implementation, this can take a substantial amount of sea icua 
time; as the team becomes familiar with the process and : 

i, ia c = Are there particular student 
the data, this will become more efficient. If EWIS data for groups your school or district has 


students entering school are available, Step 2 may be already identified as an area of 
started before the beginning of the school year. The Sechaba focus 
process of reviewing data through questions outlined in 

Step 2 is critical when using any type of early warning early 
data. For example, when it is time to review monitoring 
indicators data (Step 5), it is recommended that your team 
use this same approach to reviewing data. The following 


represents some guidelines for how teams may want to approach the data analysis process. 
Anticipated Outcomes for Step 2 


The following outcomes are anticipated for Step 2: 


= An understanding of the nature and the scope of district and schoolwide student EWIS risk levels 
(i.e., the percentages of students at low, moderate, and high risk) 


= An understanding of patterns associated with risk levels for students across the district and within 
and across schools (e.g., how risk levels vary by grade, gender, disability status, race or ethnicity, 
and other characteristics) 
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= Identification of individual and/or groups of students who show signs of risk of missing desired 
academic milestones 


= An understanding of patterns across a district, schools, and groups of students that allow the team 
to consider how to best allocate and direct interventions and supports to best meet student needs 
at the individual, group and schoolwide levels 


= Initial hypotheses related to potential underlying causes of student need 


= Tentative decisions about the scope and the focus for exploring underlying risk and identifying the 
types of additional information that the team may want to collect 


Exploring Data Through Questions 


Teams will want to begin by generating EWIS reports. It is important to recognize that EWIS (and, later 
on, monitoring indicators) reports can quickly yield a lot of information. As a result, it is very easy for the 
team to become overloaded with data and subsequently not be able to make sense of how to use the 
data or even what the data are telling them. As shown in Exhibit 6, teams can make the data exploration 
process more manageable by using three levels of examination: 


= Get the big picture. Examine data by broad dimensions and main categories. 
= Dig deeper. Examine data by subgroups or subcategories. 


= Deep dive. Examine data for individual students. 


This multilevel examination can provide a context for understanding student needs within a district or a 
school and can help the team break down the information 
into manageable pieces that can be sorted, organized, 
and prioritized so that the team can take action. (See 
Tools 7 and 8 below for worksheets to guide your team’s 


data review.) Notes from the Field 


We learned that it is best to start 
with the data and the questions 
we have rather than starting by 


saying we are implementing 
“EWIS.” This way it doesn't feel 
like something new—or one 
more thing to do. 
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Exhibit 6. The Multilevel Approach to Exploring Data 


Big picture: Broad dimensions or main categories 


Dig deeper: 
Subgroups/subcategories 


Deep dive: 
Individual 
tuden 


Get Ready. Before getting started, the team develops a plan for how they will review their EWIS data 
(e.g., which questions to answer, how to focus analyses [whole school, subgroups, and individual 
students], and who will be responsible for reviewing which reports). For most districts and schools, it is 
not feasible to fully analyze all of the EWIS data; therefore, it is helpful to have specific goals and 
questions in mind to more clearly define the task. 


Get the Big Picture: Examine Data by Broad Dimensions. The team may want to begin reviewing its 
EWIS data by generating data reports that offer the big picture of student risk within a district or a 
school, such as the EWIS District View report (EW301) or Postsecondary District View (EW303). 


It is important to acknowledge that in some schools, a high percentage of students may be identified as 
being at risk. In these schools, the team must decide how best to proceed in understanding and 
addressing student needs and how to allocate time with respect to understanding and addressing the 
needs of specific groups of students and individual students. 

Examples of broad dimensions (main, single categories) for reviewing EWIS student risk levels by: 

= School 


= Grade level 


= Academic milestone 
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Some big picture questions are as follows: 
= What proportion of the district’s or school’s students are at low risk? Moderate risk? High risk? 


= Are there differences across schools serving ; , 
Implementation Tip: 
Start with EWIS District View Report to 


= What student groups may warrant further get a Big Picture of Your Data 
exploration? 


similar grade levels? 


Edwin Analytics EWIS District View Report 
(EW301) is particularly helpful for illustrating 
the big picture of student risk for your school or 
district. By broadly examining EWIS data, the 
learned so far and what questions still need to be team is able to assess the nature and the scope 
answered in the next phase of analysis. of student risk levels within the overall student 
population of interest, before delving more 
deeply into the needs of specific cohorts of 
students or individual students. Using the 
district view report offers the team a context in 
broad landscape (overall nature and scope of need), it which to understand the extent of student risk 
is time to dig deeper into the data to uncover patterns | within the district or school, as well as inform 
decisions about the future allocation of time 
and resources. 


Before moving on to the next level of analysis (dig 
deeper), the team summarizes what has been 


Dig Deeper: Examine Data by Subgroups or 
Subcategories. After the team has a sense of the 


of risk by each subgroup of students (e.g., gender, race 
or ethnicity, ELL status, special education status, grade 


level), and patterns of students by subcategories of 
risk (EWIS student risk levels, academic milestone, or by data used to determine EWIS student risk levels). 
This deeper analysis of EWIS data helps the team identify patterns associated with EWIS risk levels for 
particular subgroups of students across the district and across and within schools (e.g., how EWIS risk 
levels vary by grade, student subgroups). The team may also 
want to explore patterns associated with particular data 
points that were used to determine the EWIS risk levels (e.g. 
How many students flagged as EWIS high risk failed 
mathematics last year? What is the attendance pattern for 


Notes from the Field 


our high and moderate risk students?). During the Di 
We worked as a team to look at . ; ; ) ; 5 : : 
the data. First, our principal Deeper phase, the team begins to identify sub-populations 


pulled the data and did a cursory of students of concern or gathers more information about a 
analysis of trends. Next our team student group already identified. 
decided which subgroup to 

target, and we pulled info on 

those students so we could look 

at individual students. Next we 

asked how many high risk 

Students are already assigned 

an intervention? For those 

students who weren't already 

receiving any supports, we 

assigned a Staff person to follow 

up to find out more. 
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Examples of subgroups or subcategories for examining EWIS data include: 


Patterns of EWIS risk levels across and/or 


within student groups: 


Student demographic characteristics 
Student language and learning needs 


Within each grade, age group category (e.g. 
late elementary) 


Patterns across data points used to determine 
EWIS student risk levels: 


Attendance patterns 
Suspension patterns 
State assessment results 


Course perfo rmance patterns 


Examples of subgroup or subcategory questions 


include: 


District level. How do student EWIS student risk 
levels differ across schools? For example, is the 


breakdown of low, moderate, and high risk 
consistent across grade 3 in our elementary schools? Are there EWIS student risk level differences 
among student subgroups, attendance patterns, etc. across the schools in the district? 


Implementation Tip: 
Take Advantage of the EWIS Report 


Options in Edwin Analytics 


Three of the EWIS reports in Edwin Analytics 
may be particularly helpful for the “dig deeper” 
level of examination: 


Subgroup Analysis (EW318) shows the 
breakdown of EWIS student risk levels by 
student subgroups, as well as a state 


comparison for the student subgroup 
populations 


Risk Level Indicator Analysis Reports 
(EW317 or EW319) provides attendance, 
behavior, state assessment, and course 
patterns for students in each EWIS student 
risk level for a selected grade level (and 
academic outcome in EW319) 


Graphical View (EW302) includes two line 
charts that provide two different 


laAnaitiudinal wiavre far tha calactad 


School level. What are the characteristics of students who have been identified as being at risk? 
What percentage of these students are ELLs, special education, and/or economically disadvantaged? 
What are the suspension patterns of students identified as high risk across the grade levels at our 


school? 


Important Note: For some districts and schools, a high percentage of students may be identified as 
being at high or moderate risk. In these cases, after districts and schools understand patterns associated 
with risk, they may choose to focus most of their efforts on addressing student needs on a schoolwide 
basis or providing supports for targeted subgroups of students. 


Before moving on to the next level of analysis (deep dive), the team summarizes what has been learned 
so far and what questions still need to be answered in the next phase of analysis. 
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Deep Dive: Examine Data for Individual Students. By now, 


the team should have a clearer understanding of which 
groups of students are in need. The deep dive requires that 
the team look beyond EWIS student risk levels (low, 
moderate, or high risk) to begin to examine the potential 
factors that may be contributing to the student’s risk for 


not meeting key academic milestones. 


Teams can begin by using Edwin Analytics to examine the 
specific data points (e.g. attendance, course performance, 
suspensions) that contributed to a student being flagged as 
moderate or high risk. For example, the EWIS student list 
report (EW 601) and EWIS Postsecondary student list 
(EW602), provides all of the data points that were used to 
provide an EWIS student risk level for a given student. For 


Notes from the Field 


Our team took EWIS and sorted 
into four different data files (by 
grade) and proceeded to color- 
code each grade level based 
upon students who had: already 
dropped out, had SPED 
services, had outside services, 
or who are in alternate 
programs. Anyone not color- 
coded and in the high risk group, 
were then the kids who had no 
one monitoring them. These 
were the kids who could possibly 
fall through the cracks. 


Notes from the Field 


We used EWIS data to gain the 
big picture on freshman 
repeaters. We dug deeper to 
identify that there were ELA 
course scheduling issues for 


many students. In addition, we 
did a deep dive to talk about the 
needs of individual students by 
providing guidance counselors 
with data on their assigned 
students. 


one student 
considered 
high risk, the 
team may find 
that the student failed math the previous year and 
attended school less than 90 percent of the school year. 
While these data points do not necessarily explain why a 
student is at risk, uncovering this additional information 
can help the team begin to formulate hypotheses that will 
guide the gathering of additional data to improve the 
team’s diagnosis of the reason a student is in need and 


thereby make decisions about interventions and supports. 


Examples of school-level deep-dive questions: 


2 Who are the individual students identified as being at 
high risk from our EWIS data? 


7 What else can we learn about these individual 
students by looking at their individual data points in Edwin 
Analytics (report EW601 and/or EW602)? 


= Do we see any patterns in the data that can guide us in gathering more information? 


Preparing for Step 3 


After the team has a sense of the overall nature and the scope of student risk based on their review of 


the EWIS data, it is time to begin considering the reasons why students may be at risk and determine 


whether and what types of additional information may be needed to gain a more complete 
SSS ae 
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understanding of the factors that may be contributing to this risk. The team should decide if that nature of 


student need warrants exploring full district and/or school changes, interventions and supports for 


subgroups of students, and/or additional study for individual students. 


It is suggested at this point that the team begin to generate initial hypotheses to explain underlying 


causes of risk and needs and articulate follow-up questions in each area of greatest concern. This 


process can be done for each of the three levels (whole school or district, subgroups of students, and 


individual students), or the team may decide to focus on only a few areas or with only a targeted group 


of students. For those districts and schools using early warning data for the first time, it is advisable to 


start small. 


Before moving on to Step 3, it is recommended that the 
team identifies its priorities, and make tentative 
decisions about the scope and the focus for exploring 
underlying causes. It also is helpful to develop a strategy 
for how the team will divide up responsibilities for 
gathering, reviewing, and interpreting additional data as 
they implement Step 4. 


District Role in Step 2 


The use of EWIS data has immediate implications for 
staff and students in a school; however, at the district 
level, there is an opportunity to examine EWIS data to 
illuminate broad trends in student risk that may 
influence how resources are allocated or policies and 
strategies are implemented to focus on issues that are 
particular to groups of students within a district. For 
example, what district-wide changes could be made to 
reduce the high numbers of incoming grade 9 students 
who are identified as high risk? 


Guiding Questions for Step 2 


Implementation Tip: 
Start Small 


Districts and schools using early warning data 
for the first time may want to learn more about 
EWIS and early warning implementation before 
attempting to serve the needs of all students 
identified as being at high or moderate risk in 
the district or the school. 


There are several ways to start small. Some 
schools may want to begin by focusing on those 
students who will be transitioning to a new 
school, such as grade 6 students entering 
middle school. Others may want to address the 
needs of a particular cohort of students (e.g., 
ELLs) or concentrate on those students who 
were identified as being at high risk. Keeping 
the scope manageable will allow the team to 
spend time refining its process so that it is able 
to operate efficiently and effectively, serving 
larger numbers of students in later years. 


= What are the overall nature and the scope of districtwide and schoolwide EWIS student risk levels 


(i.e., the percentage of students at low, moderate, or high risk)? 


= What patterns are associated with student risk levels for subgroups of students across the district 


and within and across schools (e.g., how risk levels vary by grade, gender, race or ethnicity, and 


other characteristics)? 


= Which groups of students require further digging? What are their characteristics (e.g., grade, 


gender, and ELL status)? What data points (e.g., attendance and course performance) are most 
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concerning for these students? How have patterns of risk changed over time for the district, schools, 
subgroups, and individual students? 


Given our assessment of student risk, what additional questions and information will we need to 
gather at the district, school, subgroup, and individual student levels to more fully understand the 
potential underlying causes for students’ risk and schoolwide needs? 
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Tool 5 — Step 2: Checklist 


Get Ready: 


O 


O 


O 


Confirm scope and goals for reviewing EWIS data (e.g. are there particular district or school 
priorities you want to consider when reviewing your data) 


Develop a plan for how you will review EWIS data (e.g., which questions you will answer, how you 
will focus your analyses) 


Determine who will be responsible for generating reports, reviewing report 


Get the Big Picture: 


O 
O 


Generate District View Report (EW301 and/or EW303) 


Review district or schoolwide data to determine schools, grade levels or sub groups of interest for 
further exploration 


Dig Deeper: 


Generate Graphical View (EW302), Risk Level Indicator Analysis Reports (EW317 and EW319) and 
Subgroup Analysis (EW318) reports 


Explore patterns related to student risk for sub groups of students 


Identify subgroups and individual students of concern 


Deep Dive: 


Generate Student List (EW601 and/or EW602) reports 


Review data for individual students and small sub groups of students 


Identify specific students and data related to these students that the team would like to better 
understand 


Generate Hypotheses and Set Priorities: 


Identify any schoolwide risk patterns 


Identify subgroups of students at-risk, and identify any patterns associated with risk for these 
groups of students 


Confirm the list of students and student groups the team is concerned about, and articulate the 
team’s theories about what may be contributing to risk for these students 


Establish the team’s priorities for exploring underlying causes of risk 


Identify any additional questions and information the team will need to gather to more fully 
understand potential underlying causes for students’ risk and schoolwide needs 


Tool 6 — Step 2: Practice Generating Reports 


Directions: Generate one EWIS report for each of the reporting options: District View (EW301) or 
Postsecondary District View (EW303), Graphical View (EW302), Risk Level Indicator Analysis (EW317) or 
Postsecondary Risk Indicator Analysis (EW319), Subgroup Analysis (EW318), and Student List (EW601) or 
Postsecondary Student List (EW602). 


1. District View (EW301 or EW303): Select a school in your district and generate a report for all 
grades in this school. Scan your report to find out which grade has the highest percentage of 
“high risk” students and which grade has the lowest percentage of “high risk” students. 


2. Graphical View (EW302): Select a grade level and/or student group of interest. Generate a 
graphical view report. Find out how this group compares with the overall district and state. Find 
out how risk levels for the current group of students in this grade have changed over the past 
year. Find out how risk levels for this grade, generally, have changed. Are risk levels for this 
group of students getting better? Worse? Staying the same? 


3. Risk Level Indicator Analysis (EW317 or EW319): Select a grade level of interest, and run a Risk 
Level Indicator Analysis report. What attendance, suspension, assessment results, or course 
performance patterns do you observe for this grade level? 


4. Subgroup Analysis (EW318): Select a grade level and/or student group of interest, and runa 
subgroup report to display how risk levels for this grade and/or student group vary across the 
subgroups. Which of these subgroups shows the highest level of risk? 


5. Student List (EW601 or EW602): For your selected grade and/or subgroup of students, generate 
a Student List report to view some of the individual students who are at high risk. What 
additional information do you observe when viewing the individual students? 
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Tool 7 — Step 2: Exploring Early Warning data by Broad Dimensions/Main Categories 


A. Big Picture: Here are some sample “Big Picture” questions. You may choose questions from the list below to explore, or devise your own 


questions. 


Sample Broad Questions 


How do our student risk levels vary by 
school level (elementary, middle, and high 
school), by school, or by grade across or 
within schools? 


3. How do our students’ risk levels differ by 
risk level indicator categories? 
= Attendance 
= Suspensions 
= MCAS results 
=" Course performance 

4. How have student risk levels changed over 
the past year? Over the past two years? 


5. How do risk levels differ according to 
student characteristics, across or within 
schools? 

Gender 

High-need students 

ELL students 

Students with disabilities 
Overage students 

Other 


| EWIS Data & Reports EWIS Data Output Percent or Number of Students 


What proportion of our student body is at District View Report (EW301) 
low, moderate, high risk? PS District View Report (EW303) 


ow 
ahh 
tf 
_ 
_ 


District View Report (EW301) 
PS District View Report (EW303) 


Risk Level Indicator Analysis 
(EW317) 

PS Risk Indicator Analysis 
(EW319) 


Graphical View Report (EW302) 


Subgroup Analysis (EW318) 


Use the Data Exploration Summary Chart to reflect on what you have you learned. Which grades, student groups, or indicator 
categories would you like to explore further? 
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B. Dig Deeper: Exploring EWIS Data by subgroups/sub-categories 


Directions: Identify one student group, school, or grade, of concern to explore further (e.g., middle school boys or students flagged due 
to attendance.) Here are some sample “Dig Deeper” questions for exploring the data related to your target group of concern. You may 
choose from the questions listed below, or devise your own questions. Important Note: Repeat this step for each targeted group of 


concern. 


sample 2 Suber cup/Caregery Questions EWIS Data & Reports | EWIS Data Output Notes Percent or Number of suidents 


How do risk levels for (selected 
student group) vary by additional student 
characteristics (e.g., gender, income, ELL status, 
disability status.) 


2. Within our student group of concern, how many 
individual students are identified as “high risk”? 
As “moderate risk”? 


How have risk levels for our student group of 
concern changed over time? 


Subgroup Analysis (EW318) Moderate 
Risk Level Indicator Analysis 
(EW317) 


District View (EW301) Moderate 
PS District View Report (EW303) 
— eal heel 


Use the Data Exploration Summary Chart to reflect on what you have you learned from reviewing your EWIS data. Which particular 


students are you most concerned about? 
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C. Deep Dive: Exploring early warning data by individual students 


Directions: Identify individual students of concern to explore even further.* 


Individual Student-Level Questions | EWIS Data & Reports | EWIS Data Output Notes about Students 


Looking at the refined subgroup of students 
identified, what can we learn about their shared 
risk level indicators? 

Attendance 

Behavior 

Course performance 

State assessment test performance 

Previous year’s risk level 


What can we learn about patterns in the 
characteristics of these students? 
Grade? 
Learning and language profile? 
Other demographics (e.g. race/ethnicity) 
Other student characteristics? 


Subgroup Analysis 
(EW318) 


Student View (EW601) 
PS Student View (EW602) 


Export data to Excel to 


explore shared grade/age 
or other learning or 
demographic 
characteristics for groups 
of individual students you 
are concerned about. 


Risk Level Indicator 
Analysis (EW317) 

PS Risk Indicator Analysis 
(EW319) 

Student View (EW601) 
PS Student View (EW602) 
Export data to Excel to 
explore shared risk level 
indicators for groups of 
individual students you 
are concerned about. 


Use Data Exploration Summary Chart to reflect on what you have you learned. 


*When preparing for Step 3, schools will also want to identify students by name and classroom to explore potential underlying causes for risk. 
a a a a a 


46 


Overall risk levels 
Comparison with state 
Comparison with district 


By grade 
By school 
By year 


District and/or schoolwide by student groups and 
characteristics: 


Gender 

Race/ethnicity 
Economically disadvantaged 
ELL 

FELL 

SPED 

High Needs subgroup 


District and/or schoolwide by risk level indicator 
category: 


Attendance 

Suspensions 

State Assessment performance 
Course performance 


Tool 8 — Step 2: Early Warning Data Exploration Summary Chart for EWIS Data 


Directions: Use this chart to take notes as you examine your EWIS data. 


oc na Key Findings 


= Picture: Main Categories 


Key Findings | Areasof Concern 
By student groups within and across grades: 
Gender 
Race/ethnicity 
Economically disadvantaged 
ELL 
FELL 
SPED 
High Needs subgroup 


By risk level indicator category within and across 
grades: 

Attendance 

Suspensions 

Course subject Pass/ Fail 

State Assessment test performance 


By student groups within/across other student 
groups and categories (e.g., ELL/SPED students 
with low MCAS): 

Gender 

Race/ethnicity 

Economically disadvantaged 

ELL 

FELL 

SPED 

High Needs subgroup 

Attendance 

Suspensions 

Course performance 

State Assessment performance 


Deep Dive: Individual Students 


oO oO Oo oO oO oO 0 GO oO OO @ 


Shared risk indicators and/or shared 
characteristics of individual students of concern: 
Gender 
Race/ethnicity 
Economically disadvantaged 
ELL 
FELL 
SPED 
High Needs 
Attendance 
Suspensions 
Course subject Pass/ Fail 
State Assessment performance 
Other characteristics: 
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Tool 9 — Step 2: Early Warning Implementation Action Planning Tool 


Directions: The following template is designed to support your early warning implementation efforts for Step 2. Teams can identify the key 
objective or task, and then identify the resources available, actions needed responsible parties, and when the task needs to be completed. 


Key Tasks: 
* Generate EWIS data reports. 
* Determine key questions and/or focus for data exploration. 
¢ Determine the data analysis plan (who will participate in exploring data, when, for which data?). 
* Examine data at multiple levels (Big Picture - district and/or school level; Dig Deeper - subgroups; Deep Dive - individual students). 
¢ Identify patterns and summarize key findings (e.g., complete the data summary chart). 
* Identify schoolwide/district-wide groups of students, and/or individual students of concern. 
¢ Articulate initial theories or hypotheses related to the findings. 
* — Identify follow-up questions to be answered with existing and/or additional data. 


Step 3: Explore Underlying Causes 


What You Need for Step 3 


= EWIS data and reports 


= When available: Monitoring data and reports showing student risk flags 


= A list of follow-up questions raised by the EWIS data analysis conducted in Step 2 


= Potential Underlying Causes Worksheet (See Tool 11). 


= Astrategy for how the team will divide up responsibilities for gathering, reviewing, and interpreting 


additional data 


= Sufficient time for the team to meet and review additional student data and discuss findings 


Description of Step 3 


Step 3 is about uncovering the reason(s) why students 
are at risk for not meeting academic milestones. It is 
important to acknowledge that EWIS risk levels and 
monitoring flags are merely signs or symptoms of 
deeper and likely more complex problems related to 
student achievement. Thus Step 3 builds on the review 
of early warning data by encouraging the team to 
explore potential underlying causes for individual 
students, groups of students, the school, and the 
district. Step 3 relies on a combination of data and 
professional judgment in assessing student needs. 


Anticipated Outcomes for Step 3 


The following outcomes are anticipated for Step 3: 


= Additional diagnostic data for individual students or 


Build Off Your Foundation 


= How do youcurrently determine 
the reasons why students are 
struggling? Who participates in 
this work? 

= Which data sources do you use? 
What aspects of school context 
and/or student functioning do 
you generally consider when 
investigating student needs? 

= How can you integrate efforts to 
determine the reasons for 
students’ EWIS risk levels with 
existing efforts? 


groups of students identified in Step 2, which can be used to discern the most likely underlying 


causes for risk 


= A better understanding and evidence to discern the underlying reasons that individual students or 


groups of students may be at risk of not meeting academic milestones 


= Identification of individual and common needs among groups of students and schoolwide 


= Initial ideas related to the nature or the types of interventions and supports students may need to 


help them get back on track 
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Moving From Data to Action 


As shown in Exhibit 7, Steps 2, 3, and 4 are about moving from data to action for students. In Step 2, the 
team reviewed EWIS data to identify patterns (A) and formulated initial ideas and follow-up questions 
(B). As shown in Exhibit 7, the team engages in two key tasks in Step 3: gathering and interpreting 
additional diagnostic data to identify likely factors contributing to underlying risk (C) and using these 
data as evidence to confirm likely causes of risk and the likely needs of students (D). In Step 4, the team 
takes action by making schoolwide changes, and assigning individual and group supports (E). 


Exhibit 7. Moving From Data to Action 


e D. Identify key 


e. Collect 


& B. Prepare 
hypothesis of 
key issue. 
Targets: 

en Review Group? 
data to Individuals? 
identify *Schoolwide? 
patterns. 


*Grade level? 


Step 2 == 


additional data. 


eldentify new 
data sources, 
who will gather. 

*Revisit A, 
discuss patterns 
in data. 


Step 3 


problem(s) to be 
addressed. 


*Use multiple 
sources of data 
to support this 
conclusion. 


Was hypothesis 
confirmed? 


E. Identify and 
implement 
schoolwide 
changes, and 
interventions 
and supports 
for groups and 
individual 
students. 


Step 4 
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Identifying Potential Factors Underlying Student Risk 


After the team has identified students in need (schoolwide, 
groups, or individual students), the team then explores the 
potential underlying reasons for student need by examining 
student strengths and challenges, the nature of their 
classroom or school environments, and, potentially, outside 
contextual factors. 


This exploration takes into account that student success is 
dependent on more than just academic knowledge and skills. 
The reasons why students struggle or thrive in school stem 
from a complex interplay of individual; social; emotional; 
instructional; organizational; and other contextual factors, 
such as families, 
peers, and 
communities.’ 

iS Research on 
Notes from the Field eer ne 
contribute to 


; resiliency,?® skills 
We discovered that the reasons 


for student risk are far more 


Notes from the Field 


How we went about interpreting 
our EWIS data was not 
necessarily any different than 
our approach for understanding 
student needs in the past. 
However, by using early warning 
data, the size of the pool of 
students in need has increased. 
So, the process has become 
both harder and easier: Harder 
because there’s somuch more 
data; easier in that we now have 
more pieces of data all saying 
the same thing. 


needed for the 21st century,’ and emerging research on 


complex than we may have the brain*® further support the notion that learning and 
realized in the past. student performance are dependent on the complex 


interaction between multiple domains of development and 
functioning. These, in turn, interact with contextual factors. 
For instance, five students all identified through EWIS data 
as high risk and all with attendance issues may have five 
distinct reasons (individual and contextual) for displaying 
symptoms of need and may require five different 
interventions to meet these needs. 


As the team explores the potential underlying causes for 


student risk, the team may find it beneficial to consider the range of student needs and the abilities that 


contribute to school success. As shown in Exhibit 8, individual development can be conceptualized as 
encompassing four key domains: physical, intellectual or cognitive, psychological or emotional, and 
social.** Exhibit 8 also illustrates a sample framework for identifying students’ specific strengths, 


abilities, attitudes, and functioning within each area that may be contributing to or inhibiting school 


performance. As the team explores the potential underlying causes of student need, a conceptual model 


such as this one may help the team understand the needs of students and become more strategic about 


efforts to collect additional data. 
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Exhibit 8. Domains of Student Development and Functioning 


21st century skills, 
including executive 
functioning 


Academic content, 
current knowledge 
and skills 


Physical 


Physical health and @ 
well-being 4 


Emotional or 
Psychological 


Students do not function in isolation. Rather, research suggests that student performance is the result of 
each individual’s interaction with his or her environment.” Effective learning environments are 
composed of a complex combination of factors.?? For instance, research suggests that learning is more 
likely to take place in a supportive social environment.” 
Research also suggests that learning is enhanced with 
appropriate structure (i.e., when clear rules, routines, and 


expectations are balanced with opportunities for choice Notes from the Field 
and autonomy). Students also learn best in an 

environment that offers ample opportunities for challenge We were able to add a qualitative 
and skill building, which will help students stay engaged flavor to our data interpretation by 
and foctised on their learning.*? Teams may want to ae pial iospatphorieae on 
consider all of these factors as they explore underlying had identified. Using this 

causes of student risk. Given the complexity of potential additional! data we could now talk 
underlying individual and contextual causes, the team may about individual student 


performance and we could drill 
down deeper to better understand 
teachers have the greatest opportunity to make a positive why students were at-risk by 
change in student performance. (See Tool 11, Potential looking at information such as 
their attendance, whether they 
were doing their homework, and if 
they were completing long-term 
projects. 


then want to focus on the areas in which a school or its 


Underlying Causes reference sheet.) 
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Gathering Additional Information 


To fully understand underlying factors in student risk, typically, the team needs to gather more data. 
However, before gathering further data, each team will want to be clear about its priorities, focus, and 
available time. To know where to start when collecting data, the team may want to begin by generating 
hypotheses for potential underlying reasons for risk related to its priority areas of concern—whether 
schoolwide, for groups of students, for individual students, or all three. To guide data collection, these 
hypotheses can be translated into questions the team would like to answer. For instance, a school that 
discovers a disproportionate number of grade 3 students at high risk may hypothesize that more needs 
to be done in grades K-2 to enhance students’ literacy skills. So this team might ask the following 
questions: 


= What does our K-2 literacy benchmark data tell us about student performance in key literacy skill 
areas? 


= Which students in grades K-2 need additional literacy supports—and in what areas? 
= How are we currently supporting student literacy in K-2? 


= Is our reading curriculum and appropriate for our student population? 


a“ Next, the team needs to determine the methods and the 
Notes from the Field sources of information the team will use to collect data to 
answer its questions. To manage the amount of time 
required, the team identifies those responsible for gathering 
Leave your bias at the door. this additional information. In some cases, this may mean 


You'll be surprised at what your asking others outside the team to assist. The team also 
data will show you — surprised in 


may want to establish a timeline for bringing the group 
good and bad ways. 


together to examine findings, along with additional sources 
of data in preparation for Step 4. 


Schoolwide Data Collection. During Step 2, the team 
identifies a focus, priorities, and the scope of further study. 
For instance, a school with large numbers of students in 
need may have decided that data collection will focus on 
understanding factors that influence the whole-school 
environment. This school may want to conduct a learning 


walk-through, where educators select an area of inquiry and 
systematically collect data to answer their schoolwide questions (see 


http://www.mass.gov/edu/docs/ese/accountability/dart/walkthrough/implementation-guide.pdf /) or 


conduct a focus group with school counselors. 
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Subgroup or Cohort Data Collection. Another school might discover in Step 2 that many of its students 
who are at high risk failed mathematics the previous school 


year, and monitoring flags also point to concerns with faplenientateniiin: 
implementation Tip: 
mathematics in the current school year. As a result, the How Much Data Are Enough? 


team may gather additional information about the current 
mathematics curriculum, methods of instruction, teacher It is easy to keep identifying additional 
questions and data sources that the 


quality, and the supplemental supports that are available. ; 
team would like to gather. To manage 


this process and avoid amassing too 
much data, aim for gathering three 
students in need, or where students are not responding sources of information for each student 
positively to interventions and supports, the team may or group of students and be sure the 
team is clear about what questions it 
hopes each source of data will answer. 


Individual Student Data Collection. For schools with fewer 


want to obtain additional information to help diagnose the 
specific needs of individual students. Many teams begin by 


gathering information about students directly from those 

adults who interact with these students on a regular basis (e.g., classroom teachers, school counselors, 
and specialists). For example, for students who are identified as they enter grade 1 as being at high risk 
for not being able to read by the end of grade 3 (scoring meets or exceeds expectations on the grade 3 
ELA State Assessment), the team may need to gather information on students’ kindergarten 
performance from the prior year’s teacher. Often, a team also accesses existing information that is 
available through school data systems and locate other performance or assessment data for the 
student. 


Finally, in some cases, the team may determine that it is appropriate to conduct (or work with classroom 
teachers to conduct) one-on-one meetings with individual students, their family members, or both as 
appropriate. In-person data-gathering efforts with educators, students, and families shed light on the 
reasons students are identified as being at high or moderate risk or are flagged for a monitoring 
indicator and also can spark interest and opportunities to engage students and the adults who interact 
with them in providing additional interventions and supports. If the school is implementing Individual 
Learning Plans (ILPs) with students, the ILP process is an opportunity to have a meaningful discussion 
with the student (and possibly also the family) to learn more about the student’s needs, strengths, and 
barriers. 


Most importantly, the additional information gathered during Step 3 improves the team’s understanding 
about why students are identified as being at risk. The guiding questions for Step 3 suggest a sequence 
of inquiry steps. 


Interpreting Data 


It is easy to confuse a symptom with the cause for that symptom. For instance, when a child exhibits 
disruptive behavior in the classroom, a teacher is likely to implement a behavior management plan. 
However, simply addressing the behavior itself is unlikely to be effective in the long run if the reasons 
for a child’s behavioral struggles are not fully understood. Perhaps the child is bored, is overwhelmed, 


og 


has a disability, is having difficulty understanding the 
material, or is distracted by teasing from peers. Until 
educators can systematically and thoroughly interpret the 
early warning data they have gathered so that they can P 
better understand the underlying causes for student Notes from the Field 
difficulties, appropriate interventions and supports cannot 


be devised. In addition, although individual students Wiha nie cae heise ie 


respond to stressors in unique ways, it is likely that the really struggling? Historically, we 
underlying cause(s) for one student is linked to broader had always focused on ELL and 
areas in need of improvement in a classroom, a school, or students with disabilities. 


Looking at our EWIS data we 
saw that many of our at-risk 
students weren't students with 
disabilities and were non-ELL. 
interpret what the data mean. To begin, the team reviews These were students who were 
its original EWIS reports, (and when available monitoring struggling but not receiving 


flags) for schools, groups of students, and/or individual supports. 


a district that, if addressed, would benefit many students. 


After gathering additional information, the team needs to 


students. Teams combine these data with additional 
diagnostic data that has been gathered to discern potential 


underlying causes of student need. As the team reviews 
data and conducts the analysis, it is suggested that the team openly discuss any previously held 
assumptions about individuals or groups of students and replace those assumptions with ideas about 
likely underlying causes that are based on factual evidence. 


Confirming Student Needs by Using Evidence 


During Step 3, it is likely that the team will develop new ideas for the underlying cause(s) for student 
risk. On the basis of these investigations, the team should be able to identify some common and 
individual needs among students and prepare to identify and implement appropriate intervention 
strategies (Step 4). Therefore, the last three tasks in Step 3 are to: (1) clarify and confirm that the team’s 
hypothesis is based on the collected evidence; (2) identify specific support goals for students, groups of 
students, or the whole school; and (3) establish initial priorities for how student and schoolwide needs 
will be met. It is critical to designate enough time for these tasks. Some teams may want to designate 
subgroups or work with other educators and/or classroom teachers to address particular students or 
groups of students and then reconvene as a full group to make final decisions and plan action. Before 
moving on to Step 4 it is important that the team establishes priorities for meeting student needs so 
decisions about the use of available resources needed for interventions can be made most efficiently. 


Important Note: The first time the team meets to discuss underlying causes of student risk will require a 
substantial amount of time. As the team clearly identifies sources for student information and questions 
to consider for understanding student needs, it will become more skilled at the process, so the time 
needed to conduct Step 3 will decrease. 
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District Role in Step 3 


Districts have an important role to play in supporting a school’s additional data collection efforts. Often, 
schools need to access information beyond the EWIS information housed in Edwin Analytics and/or the 
monitoring indicator system. For instance, a school may want data from its feeder schools. District 
administrators can support these efforts by meeting with teams to understand and support their local 
data collection and developing policies that give the team members or designees access to information 
so that they can efficiently and independently gather information. In some cases, the district may want 
to participate in data collection efforts or conduct its own (additional) data collection efforts. A district 
also can support its school-level teams by providing sufficient time and staff coverage to allow the 
school-level teams to engage in the time-intensive process of exploring the underlying causes of risk for 
students. 


Guiding Questions for Step 3 
= About which student(s) are we most concerned? What do we already know about them? 


= What are the team’s initial theories or hypotheses about the most likely underlying causes of 
student need? 


= Are there schoolwide concerns related to patterns in student risk? 
= What are the team’s next steps for collecting additional information? 


¢« What types of information will be most helpful in better understanding the most important 
underlying causes for the risk status of students? 


« What sources (e.g., learning walkthroughs, teachers, school counselors, assessment data, 
student interviews) will the team gather information from? 


¢ Who will collect the information and when? 


= Based on the evidence collected, what does the team believe are the most important underlying 
causes of student need? 


* For individual students 
* For groups of students 
* At the school level 
¢ At the district level 


= What are initial ideas for potential interventions and supports for the whole school, groups of 
students, and individual students? How will you prioritize these needs? 


Tool 10 — Step 3: Checklist 
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Define the Target: 

O Confirm which at-risk student(s) the team is most concerned about 

O Clarify and document what the team already knows about these students 
O Articulate hypotheses related to causes of risk for these students 


Gather Additional Data: 


O Identify what additional information needs to be collected to better understand 


underlying causes of risk for identified students 


Determine which data sources will yield the best information to set data collection 
priorities 


Determine how and who will collect these data (i.e. create a Data Collection Plan) 
O Review and analyze the additional data 
Confirm Likely Underlying Causes and Student Needs: 


O Determine what has been learned about underlying causes of risk from new data 
or evidence; confirm likely cause(s) 


O Determine what at-risk student(s) need (define the problem to be solved) 


O Generate initial ideas for appropriate interventions and supports 
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Tool 11 — Step 3: Potential Underlying Causes of Risk and Associated Data 
Sources Reference 


Directions: Use this optional reference sheet to help identify the potential underlying causes of risk for a 
student or group of students of concern. Causes of risk may due primarily to individual student, 
contextual or other factors or a combination of factors. Once you have identified potential causes of 
risk, select the data sources you will use to further explore the likelihood that these factors are 


contributing to risk for a student or students. 


= 


EWIS student risk levels 


O 
O 


O 
O 


Are social competence and relationship 
factors contributing to risk? 


Lack of social skills or competencies 
Difficulty understanding social norms 
for school and classroom 

Difficulty relating to peers 


Difficulty relating to teachers and other 


adults 


Are academic or cognitive factors Teacher conversations or 
contributing to risk? O Student data in other Edwin interviews 
O Academic skills or content knowledge reports Student interviews or surveys 
(by subject) O Monitoring Indicator Flags Family interviews or surveys 
© Transferable learning skills (e.g., O Student attendance Additional academic 
organization, problem solving, decision O Grades (current and prior) assessments 
making) O Teacher progress reports Afterschool or community 
O Learning profile (preferred learning O Grade retention program staff interviews or 
modes or methods, information O State Assessment scores surveys 
processing) O Benchmark assessment data Other 
O Language proficiency O Subject area tests 
O Special learning needs (academic, O Language proficiency tests 
cognitive, or executive functioning) O Special education testing 
O Other results 
O Other 
Are emotional or psychological factors O Teacher progress reports School counselor, social 
contributing to risk? O Special education testing worker, or psychologist 
O Attitudes and beliefs about self and results interviews or reports 
own competencies O Results of other assessments Teacher conversations or 
O. Intrinsic motivation O Suspensions interviews 
O Growth versus fixed mindset O Behavioral records Classroom or other 
O Mental health issues (anxiety, O Student attendance observation 
diagnosed mental health condition or O School counselor records Student interviews or surveys 
issue) O Medical reports Family interviews or surveys 
O Witness to or victim of a traumatic O School nurse reports Afterschool or community 
event © Doctor reports or letters program staff interviews or 
O Special needs (emotional, O Letters or notes from family surveys 
psychological, behavioral) O Other Additional assessments 


Other 
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Are physical health or well-being factors 
contributing to risk? 


O 


OoOO000 


Chronic or acute health issues 
Fatigue or insufficient sleep 
Difficulty managing stress 

Risky behaviors (drugs, alcohol) 
Physical handicap or challenge 


Classroom level 


© Subject area 

O Level of rigor of class or classes 

O Curriculum/sequence for course 

O Teacher match to students’ individual 
learning needs (e.g. personality, 
instructional approach, modality, level 
of challenge too difficult or too easy, 
pacing) 

O Relevance or match to student 
interests and goals, 

O  Student-teacher ratio 

© Size or composition of class(es) 

O Classroom match to student learning, 
social and emotional needs 

O- Time of day of class 

O Volume or type of homework 

O Peer mismatches 

O Other 

School level 

© Curriculum 

O School Schedule 

© Pace or transitions 

O Policies and procedures for matching 
students with teachers 

O Discipline policies 

O Protocols for managing students 
moving in, out, and between schools 

O Other 


OoOO0O0°0 


Teacher or classroom 
assignments 

Academic aide assignments 
Class size and composition 
Student-teacher ratios by 
classroom or grade 

Subject area level (e.g., 
remedial, standard, 
accelerated) 

Curriculum and lesson plans 
for subject or class 

School or class schedule 
Teacher website 
Homework assignments 
School policies and 
procedures 

Teacher attendance 
Classroom attendance or 
participation records 
Other 


School or classroom 
Learning Walkthrough 
Principal interviews or 
reports 

Teacher conversations, 
focus groups, interviews, or 
surveys 

Student focus groups, 
interviews, or surveys 
Family focus groups, 
interviews, or surveys 
Other 
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Pragmatic or logistical issues: 
Oo 
Oo 
Oo 


Transportation to and from school 
Class scheduling conflicts 


Lack of appropriate place or time for 
homework 


Lack of access to internet 


Outside work responsibilities such as 
caring for siblings, employment 


Challenges outside of school: 


00000 0 


Mobility 

Homeless 

Issues with peers/gangs 
Neighborhood crime or safety 
Family stressors, issues 

Other 


o00 0 


Student attendance 
School or class schedules 


Bus schedule or pick up and 
drop off locations 


Teacher website 
Homework assignments 
Mobility records 

Other assessments (e.g., 
Youth Risk and Behavior 
survey) 

Crime or police records or 
reports 

Other 


Principal interviews or 
reports 

Other school administrator 
interview or report 
Teacher conversations, 
focus groups, interviews, or 
surveys 

Student focus groups, 
interviews, or surveys 
Family focus groups, 
interviews, or surveys 
Afterschool or community 
program staff interviews or 
surveys 

Other assessments 

Other 
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Tool 12 — Step 3: Early Warning Implementation Action Planning Tool 


Directions: The following template is designed to support your early warning implementation efforts for Step 3. Teams can identify the key 
objective or task, and then identify the resources available, actions needed responsible parties, and when the task needs to be completed. 


Key Tasks: 
* Confirm students and/or target group(s) of concern. 
¢ Clarify hypotheses and identify any questions to be answered with additional data. 
¢ Identify suitable and sufficient data sources for each area, student, or group of concern. 
¢« Determine who will collect these data and when. (How the team will divide up responsibilities for gathering, reviewing, and interpreting 
additional data?) 
¢« Review existing and additional data to discern likely underlying cause(s) (i.e., root cause analysis). 
* Confirm the problem to be addressed and cite evidence to support your conclusion. 
¢ Identify student needs and support goals that the team plans to address. 
¢ — Identify initial ideas for the types of interventions and supports that will be needed. 
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Step 4: Take Action 


What You Need for Step 4 


= An understanding of student needs in the school, based on evidence gathered by reviewing EWIS 
data and monitoring flags as well as additional information gathered in Step 3 


= Identified support goals for students, groups of students, and/or schools 
= Initial priorities for how student needs will be met 


= If applicable, information regarding your district/school’s procedures for implementing a tiered 
system of support 


= Tools 14 and 15, Interventions and Supports Inventory and Review 


= Appendix D: Targeted and Schoolwide Interventions Resources 
Description of Step 4 


Step 4 is about taking action. In this step, using the data 
from Step 3, the team identifies appropriate interventions 


and supports to align with the needs of students on an Build Off Your Foundation 
individual, group, and/or schoolwide basis. During Step 4, = How do you currently identify 
the team creates an inventory of available academic and interventions and supports for 
oe : : students? 
non-academic interventions and supports in the school, Do Hosed a 
the district, and the community and identifies and pene” ee ee 
addresses any gaps in those supports. Next, the team « Which current teams are 
considers how best to allocate and assign resources to involved in identifying and 
meet student needs, which can be approached through a providing interventions and 


supports? In making 
schoolwide improvements? 
= How can you integrate the way 
supports, the team may draw on available resources, you provide supports and 
interventions to students and 


: . settee address schoolwide 
schoolwide changes. Finally, the team works with district improvements? 


tiered continuum of academic and nonacademic supports. 
As the team identifies appropriate interventions and 


adopt and offer new programs, and/or implement other 


and/or school educators and students to ensure that there 
is sufficient intensity, oversight, and commitment for the 


interventions and supports to be successful. 


Anticipated Outcomes for Step 4 


The following outcomes are anticipated for Step 4: 


= Acompiled inventory of academic and nonacademic interventions and supports available to 
students within and outside of the school 
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= The identification of gaps in the available interventions and supports for students and 


recommendations for new strategies 


= Information regarding potential evidence-based interventions and supports for identified student 
needs (available or desired) 


= The provision of interventions and supports on the basis of the student needs identified in Steps 2 
and 3 (documented for each individual student). Districts/schools may integrate this with their 
approach to assigning universal, Tier II, and Ill supports and interventions through their tiered 
system of support 


= Aplan for tracking and assessing the success of interventions and the fidelity of implementation 
= Aplan for monitoring student progress and adjusting interventions and supports 


= Recommendations for grade, schoolwide, and districtwide core or universal changes and support 
strategies aimed at addressing the most common student needs identified in Steps 2 and 3. 
Districts/schools may integrate this with their approach to Tier | interventions through their tiered 
system of support 


Using a Systematic Approach to Assigning Interventions and Supports 


In many schools and districts, interventions and supports for students are available but could be more 
systematically delivered and better coordinated. The use of EWIS data and monitoring indicators can 
be the foundation for developing or furthering a systemic approach to interventions. Schools and 
districts are increasingly organizing specific strategies or interventions into a continuum of supports 
that is based on the needs of the students and the intensity and the duration of the interventions or 
supports. One model of this approach is the Massachusetts Tiered System of Support (MTSS), which 
outlines a blueprint that is responsive to the academic and nonacademic needs of all students. MTSS 
offers a structure to provide students with access to high-quality core educational experiences ina 
safe and supportive learning environment and a continuum of services that provides increasing levels 
of support. The three tiers of MTSS are as follows: 


= Tier | interventions and supports are the general education environments that are safe and 
supportive, the curriculum is rich and rigorous, and the instruction is designed to meet the 
variability of learners. 


= Tier Il interventions and supports are moderately intensive and are targeted toward small groups of 
students with similar needs who would benefit from similar interventions or supports (and 
sometimes individual students). 


= Tier Ill interventions and supports are the most intensive and are provided to individual students 


with the highest levels of need. 


Exhibit 9 is a graphical depiction of the flexible tiers for providing interventions 
and supports within MTSS. For more information on MTSS, see http://www.doe.mass.edu/sfss/ . 
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Exhibit 9. Flexible Tiered Approach to Student Support 


Tier Ill 
Intense and core 
instruction 


Tier Il 
Supplemental and core instruction 


Tier | 
Core instruction and universal behavior supports 


When adopting or adapting such a model for keeping students on track, schools and/or districts may 


consider the following: 


Clearly communicate the purpose of and define the tiers within the model so that stakeholders— 
including administrators, educators, families, specialists, school support staff, outside agencies, and 
students—easily understand the tiers and their flexibility.? 


Establish a protocol that enables students to move through the tiers seamlessly and efficiently as 
needs are identified and then change. 


Ensure that more intensive interventions and supports do not result in a label’ for students but 
result in actionable information that leads to appropriate support. 


3 For example, a student assigned intensive 1:1 academic support to reach a certain skill level, might be moved to a small group 
intervention over time. Eventually, that same student may not need either support, but, when he/she moves to a new grade or 


school, might be reassigned to more intensive or targeted supports. 


4 Teams should ensure that students are not designated as Tier |, Il, or Ill students; rather the interventions and 
supports are Tier I, Il, or Ill. For example, students may receive universal supports (Tier |) for most aspects of the 
their education, at the same time as receiving Tier Il supports for math and Tier III supports for mental health 
services. 


It is assumed that those schools or districts using a tiered approach enables all students to have access 
to Tier | instructions and supports: broad, school, and/or districtwide changes and supports. Students in 
need also may receive Tier II or Tier III interventions and supports, or both, based on the assessment of 
need determined by a variety of factors and the team. In Step 5, the team monitors student progress 
and adjusts student assignments to interventions and supports as well as their movement through tiers, 
as needed. These features help schools and districts coordinate services and closely track the 
participation of individual students in interventions and supports and their responses to those supports. 
Important Note: Schools and districts that are not using a tiered model of support can still make good 
use of EWIS and monitoring data with their own systems of support for students in need. 


Creating an Inventory of Available Interventions and Supports 


Before providing interventions and supports to meet 
student needs, the team needs to identify what is already 


available and better understand the needs those 
Notes from the Field interventions and supports are intended to address. By 


creating an inventory of existing programs, services, 
interventions, and supports, along with the personnel 
We started out by doing a 
comprehensive inventory of all 
the interventions we offered at 
all grade levels. When we resource on which the team draws as it identifies students 
reviewed this list we saw right in need. Some districts and schools may already have 
away that we had virtually 
nothing for middle school, and 
that our interventions were being 
thought about and delivered in from a variety of departments and other sources. Tool 14 
silos by ELL, SPED, etc., rather provides a sample way to organize this information. 

than addressing the broader 

needs of students. When conducting an inventory of existing interventions 


who will provide them in the school, the district, and, if 
possible, the community, the inventory is a valuable 


information about available interventions and supports 
and may need to locate and combine this information 


and supports, the team reviews available interventions 
and supports (see Tool 15). The following questions help 
teams to organize and review these interventions and/or supports: 


= What is the nature and the purpose of each intervention and/or support? What grades or age 
groups does each intervention serve? 


= Which student needs is each intervention and/or support primarily designed to address (e.g., 
academic, learning skills, social, emotional, attitudes and beliefs, behavior, health, student disability 
and/or learning profile, language proficiency)? 


= What is the intensity of each intervention or support (e.g. Tier |, Il, III)? 


= What is the duration of each intervention or support? 


71 


= Where and when is each intervention and/or support 
being offered (e.g., which classrooms, schools, locations 
within the district or the community)? 


= What is the current capacity of teachers, administrators, 


Notes from the Field 


or specialists for implementing each intervention or 
support for identified youth? 


= Are any of the students who have been identified as We developed standard 
being at risk already participating in these interventions protocols to use across the 
or supports provided outside the school day? schools to assign Students to 
support tiers in a consistent 
= Are the interventions and supports evidence based and fashion based on relevant and 


known to be effective? For which profile of students are predictive data. 
they effective? What is the evidence? 


= Are there universal screenings and assessments in place 
that are valid and reliable predictors of future 
performance that are (or could be) used to provide 
interventions? 


= What are the gaps in the currently available 
interventions and/or supports? 


Providing Interventions and Supports 


After the inventory is developed, the team needs to prepare to take action. The team considers how 
student needs identified in Step 3 can be addressed within the continuum of existing interventions and 
supports. The team may want to begin by identifying those programs that both meet identified needs 
and are currently available on-site at the school. For some teams, the inventory of interventions and 
supports will reveal that there are few interventions and supports available that can help meet 
identified student needs. If this is the case, the team needs to determine the nature, the level, and the 
type of new interventions and supports that are needed. For a listing of resources for Targeted and 
Schoolwide Interventions and Supports, see Appendix D. This list provides information on intervention 
types, the appropriateness of interventions by school levels, and the level of research review and rigor 
of evidence for listed interventions. 


Often, the team begins by focusing on Tier | instruction and supports that can serve the greatest number 
of students in need, before assigning more intensive interventions or supports to groups of students and 
individual students. For instance, should a school adopt a new instructional approach, such as project- 
based learning, to help engage students and align more effectively with students’ learning styles? Are 
there school or district policies and practices that will help address student learning needs, such as new 
behavior management policies? Step 4 relies heavily on the data collected during Steps 2 and 3 to 
inform action, but team members are ultimately charged with using their professional judgment to 


eee 
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evaluate data and upon which recommend specific interventions and supports.*© Remember, teams can 
always adjust and add interventions and supports throughout the year. 


To help track student progress, it is recommended that the team document any small group or 
individual intervention and support assignments. As students receive interventions and supports, the 
team can monitor student progress and adjust the intervention or support on the basis of this 
information (see Step 5). 


To ensure that each intervention or support is appropriate and effective, the team must: 


= Secure leadership buy-in and support for intervention and support strategies to assist students. 


= Collaborate with and secure buy-in and time from classroom teachers, school counselors, special 
educators, and other specialists to learn how to design, implement, and oversee interventions and 
supports. 


= Ensure that planned interventions and supports will provide sufficient intensity and that 
participation or attendance in interventions and supports is documented. 


= Secure buy-in from students and families (as appropriate). If the school is implementing Individual 
Learning Plans (ILPs) for students the ILP process is a place to discuss possible intervention and 
supports. These interventions and supports can also be documented in the ILP instrument. 


= Devise a plan for tracking individual student responses to the interventions or supports. For 
example, use monitoring indicators to identify whether students in interventions continue to be 
flagged (Step 5).1” 


District Role in Step 4 


The perspective of the district allows longer-term solutions Notes from the Field 
and strategies across schools. The district can do the 

following to support its teams in identifying appropriate ‘ 

ih ee Meer It's not always about the 

te CunOrS aia Supnou student when they are high risk. 


We used EWIS data to go to 
some teachers with high failure 
rates and ask them to tell us 
"Pinpoint districtwide solutions for common needs about what's happening in their 
classrooms. We were able to 
suggest shifts in instructional 


= Identify common needs of students across multiple 
schools. 


across schools. 


= Provide solutions for gaps in available interventions and practices, use more formative 
assessments and embed 
supports. ; : 7 
instructional coaches in 
= Allocate and/or reallocate resources on the basis of classrooms to offer more 
identified needs within specific schools. Support for some teachers. 
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= Support interventions to improve student readiness for key transitions (e.g., elementary school to 
middle school, middle school to high school, summer bridge programs). 


Districts also have a role to play in identifying and promoting the use of promising intervention and 
support programs. By working with and as part of teams, district personnel can play key roles in 
identifying promising interventions and supports and share effective practices across schools. 


Box 3. Examples of District Activities 


The following are some specific activities that districts can use to support schools in Step 4: 


= Develop policies, practices, and procedures for implementing a tiered system of 
support. 


Ensure that parents receive regular updates of their children’s interventions, goals, and 
progress. 


Demonstrate awareness and sensitivity to cultural, linguistic, and other aspects of 
family diversity. 


Engage community partners and providers with the school and families to provide 
access to social services and health, social, recreational, and supplemental educational 
services. 


Promote equity by distinguishing among the academic and/or nonacademic needs of 
individual schools and populations and allocating adequate resources to the schools 
and students with the greatest needs. 


Implement a review process to determine the cost-effectiveness of its programs, 
initiatives, and activities as it relates to student achievement. 


Actively seek ways to leverage resources and expand capacity through collaboration 
with external and community partners. 


Create a climate conducive to adult learning through effective communication, ongoing 
professional improvement, and joint responsibility for students’ learning. 


Offer professional development programs and services based on district priorities, 
information about staff needs, student achievement data, and assessments of 


academic and/or nonacademic practices and programs at each school. 
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Guiding Questions for Step 4 


What needs has the team identified for individual and groups of students? Schoolwide needs? 


Do trends in the data identify the need for particular types of interventions and supports (e.g., 
professional development for teachers on instructional strategies, transition supports, or 
opportunities for extended learning beyond the school day)? 


What interventions and supports are currently implemented in the school? In other locations in the 
district? How successful do they seem to be? What changes could be made in curriculum and 
instructional practices? What additional professional development is needed for teachers? 


What evidence based interventions and supports are available to support identified student needs? 


What policies and structural quality features (as opposed to specific programs) currently exist (or 
could be implemented) to support students’ academic and nonacademic needs (e.g., flexible 
scheduling, behavioral support, or attendance and truancy policies)? 


Which interventions and supports can the team identify that, if implemented, would serve the 
greatest number of students? 


If a tiered model is not already in place, is this an approach our team wants to use to provide 
supports? Are there other ways to coordinate services and prioritize the allocation of resources? 


Do the demographic characteristics (e.g., disability status, economically disadvantaged status, or ELL 
status) of the students identified as being at risk inform intervention and support decisions? Should 
they? 


To 


Tool 13 — Step 4: Checklist 


Confirm Student Needs and Priorities: 


O 


O 
O 


Confirm the list of individual and groups of students to which your team will 
assign interventions and supports 


Identify common needs across individuals and/or across groups 


Identify any schoolwide needs your team plans to address 


Inventory of Interventions and Supports: 


Create an Inventory of current interventions and supports 


Compare the list of interventions and supports to identified student and 
schoolwide needs 


Identify any gaps between identified needs and available interventions 


Determine what new/additional interventions (across all three tiers) may need 
to be developed, offered, or identified 


Matching Needs with Support and Interventions: 


O 


O 


Determine what approach your team will use to assign interventions and 
supports (e.g. How do you already assign interventions and supports? Do you 
have a tiered model of support in place? If not, is this an approach the team 
wants to explore?) 


Determine the broad policies and structural quality features (as opposed to 
specific programs) that could be implemented to support students’ academic 
and nonacademic needs 


Determine which individual or group interventions and supports, if 
implemented, would serve the greatest number of students 


Assigning Interventions and Supports: 


O 


Determine which interventions and supports your team will assign—and to 
which students 


Determine who will communicate with students, staff, and parents regarding 
these proposed supports 


Identify who will oversee/manage interventions that are assigned 


Determine what data your team will collect regarding student participation 
and progress in interventions and supports 


Determine when/who will be responsible for providing these data to the team 


Tool 14 — Step 4: Inventory of Interventions and Supports 


Directions: Use the table below to develop a list (or inventory) of available supports and interventions in the school, the district, and the 
community that are available to meet students’ needs. Describe the focus of each intervention and support in column B. In column C, indicate 
the level of intensity — universal (U), supplemental (S), or intensive (1) — for which this applies within a tiered system of supports. Complete 
columns D-I to provide additional information and detail about each intervention or support to assist your team in matching the appropriate 
interventions and supports to the needs of students on an individual, group, or schoolwide basis. 


Interventions and Supports Inventory 


EX: Academic Volunteer tutors from |S Academic, social- \Math Individual / 2nd-4"" [Students failing 1°-3'? 2 days / week each 
Tutoring Mass Vol jocal university emotional small group grade math quarter 
Scholars benchmark 

lassessments. 


N 
N 


Tool 15 — Step 4: Interventions and Supports Review 


Directions: Use the table below to review and monitor each available intervention or support and use this information to (1) improve the team’s 
ability to match appropriate supports and interventions to the needs of students, (2) monitor the impact of interventions and supports, and (3) 


conduct a gap analysis of available supports and interventions to meet the needs of students. 


List and describe the intervention or support in columns A and B. In column C, indicate the level of support for which this applies within a tiered 
system of supports. Use columns D-E to provide further information about each intervention or support. Use columns F-H as the team monitors 
interventions and supports to track information about cost, evidence of improvement, and notes about the type of student needs that the 
intervention or support meets. 


EX: Academic Tutoring: 
UMass Volunteer Scholars 


Volunteer tutors from local 
university 


Academic, social- 
lemotional 


Gr 2-4 


Tutor — teacher 
communication log, 
quarterly benchmark 
assessments, teacher 
jprogress reports 


Many identified students do not 


speak English. Tutors are bi- 
lingual and speak the primary 
language of most students at 
the school. 


~N 
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Tool 16 — Step 4: Review Your Inventory of Interventions and Supports 


Directions: Now that you have completed your Interventions and Supports Inventory, explore the following questions. 


Reviewing Your Inventory — Guiding Questions: 
= Are there any interventions or supports that need to be added? Add any additional interventions or supports that are in place (e.g., districts 
supports, informal supports.) 


Thinking of students varying needs for intervention intensity (e.g. tiers) are there any areas in which you may not have adequate supports? 


Thinking of the early warning data for your students and your analysis, are there any supports that you may still need to develop? What are 
they? 


Assigning Supports — Guiding Questions: 


What is the range of interventions and supports we have available? 
What gaps can we see between student needs and available supports and interventions? 


What do we know about the proven effectiveness of these supports and their suitability for identified student 
needs? 


Which interventions or supports, if implemented, could serve a group of students? Which would serve the 
greatest number of students? 


What changes to grade- or school-level policies, procedures, scheduling, professional development, or 
curriculum could we make to meet common student needs? 


Tool 17 — Step 4: Early Warning Implementation Action Planning Tool 


Directions: The following template is designed to support your early warning implementation efforts for Step 4. Teams can identify the key 
objective or task, and then identify the resources available, actions needed responsible parties, and when the task needs to be completed. 


Key Tasks: 

¢ Confirm the priorities for student needs and support goals — particularly common needs across individual students, groups of students, 
and/or all students within the school/district. 

¢« Determine how decisions will be made about providing supports (e.g., tiered system of support). 

* Create and/or build on existing inventory of student supports (e.g., complete the interventions and supports inventory). 

¢ Review how student needs could be addressed through existing interventions and supports and identify which interventions and 
supports would serve the greatest number of students. 

¢ Identify any gaps and determine whether additional interventions and supports are needed. 

* Determine any school /district-wide policies and procedures and/or other broad curricular changes that could help address common 
needs. 

* Communicate with teachers, specialists, students, and families about the assigned interventions and supports. 

¢« Assign and document assigned interventions and supports. 


Step 5: Monitor Students and Interventions 


What You Need for Step 5 


= The most up-to-date student data from the district’s or the school’s student information or other 


database or system that will be used to monitor student progress 


= A list of students previously identified as at-risk and their assigned interventions 


= Access to additional student data sources for students who are not showing progress 


= Aninventory of interventions and supports 
= Tool 14- Interventions and Supports Inventory 


= Tool 15 — Interventions and Supports Review 


Description of Step 5 


Monitoring indicators are both research based and locally 
determined sources of student data available during the school 
year that are used to flag students in need. Indicators such as 
attendance, academic performance, and behavior have been 
identified in research as strong monitoring indicators. Monitoring 
indicators can complement the use of EWIS data by enabling 
teams to track student needs and students’ responses to 
interventions or supports at multiple times throughout a single 
school year. 


Tool 19 in Step 5 lists a set of commonly used monitoring 
indicators by age group and grade. Each data source and monitoring 
indicator threshold, or the point at which a student is flagged as in 
need, is determined when a research base is available. These 
indicators are shown to be research supported and marked as such in 
the fourth column of Tool 19. In instances where an indicator has 
been used in practice but there is no consensus ona threshold, or 
if the threshold is dependent on the instrument used (e.g., 


Build Off Your Foundation 


= How do you monitor student 
progress? 

= Which sources of datado you 
use? Which data systems do you 
use to organize this information? 


= How can you integrate EWIS 
data into your current data 
systems? 

= How can you align your review of 
monitoring indicators with the 
timing and data work being done 
by existing groups? 


benchmark assessment), then the fifth column in Tool 19 shows that the indicator must be locally 


determined by the school or district. 


It is recommended that teams decide before—or early on in the school year—which student data 


sources they will use to examine student risk throughout the year, set thresholds that will “flag” 


students as being at-risk, and set up a data system that can be used to monitor students. 
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In Step 5 teams assess student progress, improve the effectiveness of interventions and supports, and 
identify any new students that are flagged as at-risk. The team uses monitoring indicators data to see 
whether students are beginning to get back on track (i.e., not flagged for a monitoring indicator), or 
whether they are still showing (or newly showing) signs of risk. For students flagged as at-risk, the team 
will assign, re-assign, and/or adjust interventions and supports, as needed. Monitoring flags are also 
used to observe any students, who were not originally identified as EWIS high or moderate risk, who 
may be showing symptoms of need. 


During Step 5 the team generates and reviews up-to-date information on student risk. As part of 
monitoring, teams revisit Steps 3 and 4, Explore Underlying Causes of Risk and Take Action so that teams 
can interpret the meaning of risk flags, and adjust or match students with new or different interventions 
and supports. Step 5 also helps the team examine trends in student responses to assigned interventions 
that may allow a district or a school to assess the effectiveness of certain interventions and supports for 
certain students or groups of students. Finally, the regular monitoring of students and schoolwide 
interventions and supports allows the team to engage in planning for summer and other extended day 
programs and supports using data. 


Anticipated Outcomes for Step 5 


The following outcomes are anticipated for Step 5: 


= Identification of students who are newly flagged as at-risk, or who are continuing to be flagged as 
at-risk 


= Knowledge about individual student, subgroup, and 
whole school progress in addressing student risk 


Notes from the Field 


= Identification of gaps in the available interventions and 
supports for students, recommendations for new 


intervention strategies, and prioritization of new : 
We met once or twice a month to 


look at data so we could monitor 


= Increased awareness of how well interventions and Student progress. We spent a lot 
of time interpreting the data 


interventions and supports based on monitoring flags 


supports are working, and which interventions and 


because there were surprises. 


supports appear to be most effective in helping students 
get back on track 


= Enhanced understanding by appropriate stakeholders 
about student needs, the impact of existing 
interventions and supports, and the need for additional 
interventions and supports, if applicable 
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Preparing to Monitor Students and Interventions 


During Step 1: Get Organized, teams prepare to use early warning data, including research-based and 
locally determined monitoring indicator data before or during the early part of the school year. These 
steps need to be completed prior to monitoring students and interventions: 


= Select which student data sources the team will use to monitor student risk throughout the year. 


= Establish thresholds to “flag” students as being at risk using these data sources (e.g., students who 
miss 10 percent or more of school days). 


= Establish a system or tool your team will use to monitor students’ early warning data. 


= |f using a data system or tool, make sure that at least one member of the team has proficiency in 
importing, updating, and generating reports using the 
monitoring indicators database system or tool. 


= If using a data system or tool, establish a data import ; 
and review schedule for analyzing data and reports with Notes from the Field 


up-to-date student monitoring indicators data By monitoring students we found 
out that the intervention worked 
for 16 out of 18 students — and 

= Verify that all members of the team understand the data that they showed increased 
literacy scores and scored higher 
on other benchmarks. The data 


throughout the school year. 


sources for monitoring student progress and the various 


monitoring indicator reports. also showed that giving these 
students a fresh new start did 
Teams can select from one or more monitoring indicators. nothing to affect their attendance - 


for some it even got worse...For 
those who weren't absent and 
who were able to work directly 
points during the school year. To review or revise your with a literacy coach, they we able 


monitoring indicators, please see Tool 19 below. It is fo move much more rapidly and 
get up to grade level. 


As previously mentioned, a monitoring indicator draws 
from readily available data that can be observed at many 


recommended that the team identify or establish a system 
before, or early on in the school year, to manage the data 


that will be used to monitor student risk flags. The team has 
various options for managing these data. For instance, districts and schools may wish to use their own 
data systems for monitoring students. Or, districts and schools may choose to other available systems 
for monitoring students, such as AIR’s National High School Center EWS High School Too! or EWS Middle 
School Tool® which is available for free or other commercially available monitoring indicator tools. 


Data Import and Monitoring Schedule 


A key aspect of monitoring is the timely access and review of student data to determine which students 
are “flagged” as at-risk, and for which indicators students are “flagged”. The pre-set thresholds 
associated with monitoring indicator data sources are grounded in research that specifies particular 
timeframes for these thresholds. For example, the research on the relationship between attendance and 


lt 
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student risk found that students who missed more than 10% 
of instructional days during any given marking period were 
more likely to drop out of high school. Given these research- 
based timeframes, typically, teams examine monitoring 
indicators following each grading period. In addition, many 
teams examine student attendance after the first 20-30 days 
of school—another timeframe identified through the 
research. 


The team and the data designee should work together to 
establish a data import and review schedule for analyzing 
data and reports with up-to-date student monitoring 
indicators data throughout the year. It is important to leave 
sufficient time between the end of each grading period and 
the scheduled team meetings so the team’s data designee 
has sufficient time to import, analyze and generate reports 
for the team to review to track student progress. 


Monitoring Student Progress in Interventions 
and Supports 


Notes from the Field 


EWIS data is a good starting 
point, but once the school year 
Starts, that data is changing — 
ie., how a student is once the 


school year starts might be 
different. We found that it was 
really helpful to keep looking at 
how students were performing. 


The team can use the following strategies to monitor student progress: 


= Have a Data Designee. Each team should designate at least one member who has data access, and 
who has the proficiency and the time to take primary 


Notes from the Field 


responsibility for importing, updating and generating 
grading period reports that will be used for monitoring 
students throughout the year. In addition, it is 


We have done a complete 180 
degree turn in how we are using 
data this year. We were using 
data in an autopsy format all 
along — the data was rich, but we 
were waiting for State 
Assessment results to come out 
to say "What did we do wrong?" 
Now, we see student status, 
their progress, and adjust 
Student supports RIGHT NOW 
instead of waiting until after kids 
have dropped out or repeated a 
grade. 


recommended that the data designee take the time to help 
all team members understand the data sources being used 
to monitor student progress, and how to accurately 
interpret monitoring indicator reports. 


= Generate up-to-date information on risk. A team needs 
current monitoring indicators data before meeting to track 
student progress. In Step 1, the team establishes a system 
to use monitoring indicators and establishes a schedule 
and a person responsible for importing student data at 
these specific time points e.g. attendance and course 
performance data after each grading period. 


= Review current monitoring indicators data to identify 
students who are flagged as at-risk. For the monitoring of 
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individual students and groups of students, the review of data should focus on two sets of students: (1) 
students who were previously at risk and assigned to one or more interventions or supports, and (2) 


students who were NOT previously identified as at risk, but are now showing risk flags. The review of 
data for previously identified students should determine student progress (i.e., Have students improved 
or do they continue to be flagged as at-risk?). For example, if a group of students identified as being at 
high risk for failing their 9"" grade classes were assigned to an afterschool tutoring program, do first-term 
grades for these particular students show any improvement? See Tool 20 in Step 5 for a worksheet on 
exploring your monitoring indicators data. 


Many teams review monitoring indicator data for all students each month or following each grading 
period. This monitoring should be done as part of the team’s routine review of student data and 
incorporated as a regular item on the team’s agenda. How and how often this monitoring is 
conducted depends on the level and the nature of interventions and supports that were 
implemented in Step 4. Keep in mind that students participating in Tier 3 interventions should 
monitored frequently. 


Identify student needs that are not being met—and explore underlying causes for signs of risk. 
Students who are receiving interventions and supports may continue to show signs that they are at risk 
for not meeting academic milestones. It is possible that their needs are not being addressed by the 
interventions or supports for any number of reasons. Regular monitoring allows these issues to be 
identified quickly. The team may need to collect additional information—particularly data related to 
student participation in assigned interventions and supports to understand why an intervention ora 
support is not working. After Step 5, the team is likely to revisit Step 3 for students who are not showing 
progress or who are newly flagged at risk to identify the nature of unmet student needs, and likely 
underlying causes of risk. 


Adjust or assign new or additional interventions or supports to meet student needs. The team may 
need to identify new or additional interventions or supports that are not currently available to 
students identified as being at risk. To accomplish this, the team should revisit Step 4 activities (i.e., 
conduct a search and update the list of available interventions and supports that may serve 
identified student needs, and assign the appropriate interventions or supports). In some cases, the 
team is able to find existing interventions and supports that meet student needs; in other cases, 
new interventions and supports will be needed or broader, schoolwide changes will be 
implemented. Any new interventions and supports should be tracked and captured on the 
interventions and supports inventory (created in Step 4). 


Communicate with educators, specialists, students and families. As noted in Step 4, teams will 
want to communicate with educators, specialists, students, and their families to best support 
students who are at risk for not meeting academic milestones. Educators, specialists, and families 
should be given guidance on how they can best support these students and be informed when 
students appear to make improvements and when there is a lack of improvement or further decline. 


Review current monitoring indicators data to track schoolwide interventions. For monitoring 
districtwide and schoolwide interventions and supports, the team may want to identify which 
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indicators or other data sources to track schoolwide progress. For instance, if anew mathematics 
curriculum has been introduced, the team may want to examine midterm mathematics grades. 


= Examine the effectiveness of the interventions or 
supports. In addition to tracking the progress of 
individual students and schoolwide interventions, Step Notes from the Field 
5 also encourages the team to better understand the 
effectiveness of particular interventions and supports. 
Long term, this process improves the team’s We plan to look over the 


understanding of which interventions and supports are summer at our high risk kids wh 
did great and figure out what 


circumstances helped them, 
enables the team to strengthen the process of what worked? 


best at addressing particular student needs and 


matching students to the right interventions and 
supports (see the district role for Step 5). 


District Role in Step 5 


The monitoring of individual students and groups of 
students occurs at the school level, but the district can 
play a key role in enhancing the availability of appropriate 


interventions and supports. For example, a district can 
monitor specific interventions and supports in which students participate and identify the scenarios in 
which these interventions and supports seem to work 


Implementation Tiv: to get students back on track. The district also may 


Planning Supports for the Summer examine whether a specific intervention or support 
works well for certain groups of students (e.g., students 
The team needs to meet multiple times who are failing ELA classes or ELL students at risk). In 
throughout the year to track student 
progress and make adjustments in 
interventions and supports. In the spring, 
the team can consider which students may fidelity of implementation is the issue or if the 


need or benefit from interventions and intervention or the support needs to be dropped from 


cases where an intervention or a support does not 
seem to be working, the district can examine whether 


supports during the summer. The team can 
then identify which students will need 
summer support, devise a plan for how to 
prioritize student needs, assess available 
summer programs and/or supports, and and supports implemented in its various schools, a 


consider resources. The team may want to district is ina unique position to understand which 
invite administrators for summer school 


the array of student interventions and supports. 


By collecting districtwide information on interventions 


i interventions an orts are most effective at meetin 
programs to meetings to help make these eprcrinlonsanc:supp 8 


assignments. the needs of particular students and subsequently 
allocate the resources (e.g., staff and/or funds) on the 


basis of student and school needs. 
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Guiding Questions for Step 5 


Do students who are identified as at-risk by EWIS data in the beginning of the year continue to be 
identified as at-risk based on monitoring indicators? 


What are the most prevalent monitoring indicators for which students are flagged (e.g., attendance, 
course failures, behavior)? 


Are students (at the individual, group, and whole-school levels) who are participating in 
interventions and supports showing improvement? If so, will these students need to continue 
participating in the interventions or supports to maintain their progress and improved outcomes? 


Are students who are participating in interventions or supports continuing to be identified as at- 
risk? 


What are the likely reasons for the continued risk for students not showing progress? 


¢ Did the team adequately assess the underlying reasons for student risk during its initial review 
(Step 3)? 


« Are students fully participating in the assigned intervention(s) or support(s)? Are there problems 
with how the interventions or supports are being implemented? 


¢ Are the assigned interventions or supports appropriate for the nature and the severity of need? 
¢ Should the frequency and/or intensity of the interventions or supports be increased? 
« Have new needs or issues emerged that need to be addressed? 


¢« What additional information needs to be gathered (Step 3) to achieve a better fit between each 
intervention or support and student needs (Step 4)? 


Are any new students flagged as at-risk that had an EWIS student risk level of low or moderate at 
the beginning of the school year? What are the likely underlying causes of risk for these students? 
Which additional sources of data should we collect to better understand their needs? What 
interventions and supports are appropriate for meeting their needs? 


With whom and how should our team communicate about student progress or concerns to 
maximize support for students in need (e.g., teachers, specialists, parents, students, administrators, 
community agency staff)? 


Are organizational or structural changes needed in the school or the district to support students? 


As interventions and supports are implemented, what happens to the nature, the level, and the 
number of students with identified needs? Has the number of students in need for each indicator 
changed since the new interventions and supports have been put in place? 


Are resources (e.g., staff time, funds, transportation) sufficient to implement the desired 
interventions and supports? If not, how might additional resources be identified? 


What are we learning about the effectiveness of the interventions the supports? 
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¢ Which interventions and supports seem to consistently get students back on track? 


¢« What interventions and supports seem to work for which students? 


¢ What interventions and supports do not seem to work for students? How might these be 


improved? Should these be eliminated? 


Tool 18 — Step 5: Checklist 


Generate up-to-date information on student risk: 


Confirm data import schedule and individual responsible for importing these data 
and generating reports for the team prior to the meeting 


Confirm the team meeting schedule allows for sufficient time for reviewing data 


Generate monitoring indicator reports for all students 


Review current monitoring indicator data to flag students: 


Review data for students who were previously flagged to assess progress (e.g., Does 
a student continue to be flagged?) 


Review data to see if new students are flagged 


Review monitoring indicator data to track schoolwide progress toward broader 
school interventions and/or improvement goals 


Revisit Step 3: Explore underlying causes and identify student needs: 


O 


O 


Determine additional data sources that are needed to identify underlying causes for 
students’ being flagged 


Interpret additional student data and identify student needs 


Revisit Step 4: Assign or adjust interventions and supports: 


O 


O 


Determine which interventions and supports your team will assign—and to which 
students (individuals, groups, or whole school) 


Determine who will communicate with students, staff, and parents regarding these 
proposed supports 
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Tool 19 — Step 5: Your Selected Monitoring Indicators by Grade Level 


Directions: Use the table below to document the monitoring indicators that were selected for use throughout the school year. Each grade level 
includes attendance, behavior, and academic performance potential monitoring indicators flags students that are on or off track. In the “Using?” 
column, check of which monitoring indicators your school or district is using this year and note the data source and selected threshold for the 
monitoring indicator. 


Notes — existing data source, 


Academic ere Threshold Research Locally 
milestone indica ie Supported Determined selected threshold, etc. 


Early Elementary 
Behavior Locally defined behavior grade, office 
discipline referrals 


Academic Locally defined performance levels on 
performance Benchmark assessments 

Behavior Locally defined behavior grade, office 
discipline referrals 


1 

2 
Academic Locally defined performance levels on 
performance Benchmark assessments 


3 


vy 
= 
Oo 
6 
is} 
~ 
iS} 
2 
x 
uy 
my 
TS 
i) 
8 
x 
uy 
N 
fe} 
a 
13) 
S 


on grade 3 ELA State Assessment 


Behavior Locally defined behavior grade, office 
discipline referrals 

Academic Locally defined performance levels on 

performance Benchmark assessments 


Additional notes about monitoring indicators for grades 1, 2, and 3: 
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Academic porch Research Locally Notes — existing data source, 


milestone sid die Thresteld Supported Determined selected threshold, etc. 


Late Elementary 


Attendance 20% or more school days missed 

Behavior Locally defined behavior grade, office 
discipline referrals 

Academic Locally defined performance levels on 

performance Benchmark assessments 


Loe 
[LSS 
[i 
a 
Behavior Locally defined behavior grade, office X 
discipline referrals 
Course Failing grade in mathematics and/or ELA Xx 
performance 
[=i 
pox | 
[ie 
Lei 
a 


20% or more school days missed 
5 


Academic Locally defined performance levels on 
performance Benchmark assessments 


= 20% or more school days missed 


Meets or Exceeds Expectations on grade 6 
ELA and mathematics State Assessment 


Behavior Locally defined behavior grade, office 
discipline referrals 

Course Failing grade in mathematics and/or ELA 

performance 

Academic Locally defined performance levels on 

performance Benchmark assessments 


Additional notes about monitoring indicators for grades 4, 5, and 6: 
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F P ial me 
Academic Adelie Research Locally Notes — existing data source, 


ae T 
milestone sid lig nrestield Supported Determined selected threshold, etc. 


Middle School 


Attendance 20% or more school days missed 
7 


a a 

ee | | 

performance 

Behavior [Suspensions offcedicipinereteras | ‘| * 

[atendance _[2O%ormeresshooldays |x| CS 

ee 

performance 

schavior [Suspensions offs dscpineefevas | «| =x ~—* 
a 
ae 
ae 
as a 


; Attendance 10% or more school days missed 


Passing grades in 
all grade 9 courses 


Course Failing one or more grading period 

performance 

Credits Enough credits for promotion to next 
grade 


Behavior Suspensions, office discipline referrals 


Additional notes about monitoring indicators for grades 7, 8, and 9: 
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Academic Potential Threshold Research Locally Notes — existing data source, 
milestone Monitoring Supported Validated selected threshold, etc. 
Indicator 


High School 


Attendance 10% or more school days missed 


Course Failing one or more grading period 
performance 


Behavior Suspensions, office discipline referrals 


Behavior Suspensions, office discipline referrals 


11 Attendance Locally defined percentage of school 
days missed 

12 Attendance Locally defined percentage of school 
days missed 


ee 
=i 
ae a an 
Lise 
Course Locally defined number of course X 
performance failures 
a ae ae 
LS 
ft 
a ee 


High school graduation 


Course Locally defined number of course 
performance failures 


Behavior Suspensions, office discipline referrals 


Additional notes about monitoring indicators for grades 10, 11, and 12: 
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Tool 20 — Step 5: Early Warning Data Exploration Summary Chart for Monitoring Indicators 


Directions: Use this chart to take notes as you examine your monitoring indicators. 


Big Picture: Main Categories — Summarize your big picture monitoring indicator findings. 


# or % of students meeting or exceeding 
the threshold (i.e., flagged as at-risk by 
monitoring indicators) 


By grade 
By school 


By student groups and characteristics: 


Gender 

Race/ethnicity 
Economically disadvantaged 
ELL 

FELL 

SPED 

High Needs 

EWIS High Risk 

EWIS Moderate Risk 

EWIS Low Risk 


By types of monitoring indicators students for 
which students are flagged: 

Attendance 

Behavior 

State Assessment performance 


Course performance 
Other 


Dig Deeper: Subgroups/Sub-Categories — Identify one student group, school, or grade, of concern to explore further based on the 
students flagged for monitoring indicators (e.g., middle school boys or students flagged due to attendance). This could be the 
same group you focused on in Step 2 when reviewing your EWIS data, or a new student group. Repeat this step for each targeted 
group of concern. 
By student groups within and across grades: 

Gender 

Race/ethnicity 

Economically disadvantaged 

ELL 

FELL 

SPED 

High Needs 

EWIS High Risk 

EWIS Moderate Risk 

EWIS Low Risk 


By types of monitoring indicators students for 


which students are flagged: 


Attendance 

Behavior 

Course performance 

State Assessment performance 
Other 


Deep Dive: Individual Students — Identify individual students of concern to explore further. 
Shared characteristics of individual students of 
concern: 


Gender 

Race/ethnicity 
Economically disadvantaged 
ELL 

FELL 

SPED 

High Needs 

Attendance 

Suspensions 

Course subject Pass/ Fail 
State Assessment performance 
EWIS High Risk 

EWIS Moderate Risk 

EWIS Low Risk 

Other characteristics: 


[e) 
oO 
[e) 
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[e) 
Oo 
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[e) 
O 
[e) 
© 
[e) 
[e) 
[e) 
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Tool 21 — Step 5: Early Warning Implementation Action Planning Tool 


Directions: The following template is designed to support your early warning implementation efforts for Step 5. Teams can identify the key 
objective or task, and then identify the resources available, actions needed responsible parties, and when the task needs to be completed. 


Key Tasks: 
* Generate up-to-date information on students (e.g., monitoring indicator data). 
*« Review monitoring data to determine which students are flagged as “at risk”. 
¢ Identify student needs that are not being met. 
¢« Examine monitoring indicator data (big picture, dig deeper, deep dive) 
* Collect additional information or data as needed to better understand underlying reasons for risk and needs (revisit Step 3). 
¢ Identify new or additional interventions to meet student needs (revisit Step 4). 
* Consider broader schoolwide/district-wide changes to meet student needs (revisit Step 4). 
¢« Examine the effectiveness of interventions for individuals and groups of students. 
* Communicate with teachers, specialists, students, and families to provide/adjust interventions and supports. 
* Conduct summer and end of year planning to support students who continue to show signs of risk. 


Step 6: Refine the Early Warning Implementation Process 


What You Need for Step 6 

= Documents describing the team’s goals, mission, roles, and responsibilities 
= List of monitoring indicators that were selected and used 

= Data on the effectiveness of interventions and supports 


= Tools 14 and 15 (provided in Step 4), the Interventions and Supports Inventory and Review 
Description of Step 6 


In Step 6, the team should reflect on the early warning implementation process. At a minimum, this 
reflection should occur on an annual basis, at the end of the school year or over the summer. The team 
discusses what has worked, what should be modified, and what should be replaced or eliminated. 
Where appropriate, any decisions about changes to the process should be supported by data and 
evidence and documented. Finally, the current team and other school and district leadership identifies 
which team members will continue to serve and who the new team members will be for the upcoming 
year. The district and the school create a plan for orienting new members and arranging for them to 
become trained in the early warning implementation process. 


Anticipated Outcomes for Step 6 


The following outcomes are anticipated for Step 6: 
= Ashared perception of the success and challenges of the early warning process 
= Clear recommendations for improving the work of the team and steps in the process 


=  Anestablished team for the upcoming school year, composed of members with a clear 
understanding of the process and their roles 


= Documentation of any planned changes (e.g., data sources, tools, personnel, policies and 
procedures) 


Evaluating and Refining the Early Warning Implementation Process 


When evaluating the early warning implementation process, it is recommended that all aspects of the 
implementation process be considered, including how the team is organized and functions. The conversation 
can be guided by exploring what is working, what is not working, and what changes will be needed to 
improve the team’s effectiveness and efficiency, as well as its ability to successfully achieve its goals for 
students. The team’s examination can range from exploring the effectiveness of certain interventions to how 
the team can access data reports in a timely and an efficient manner. To ensure that the reflection meeting is 
productive, the team leader may want to ask all members to respond to a set of questions ahead of time (see 
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the guiding questions for Step 6). This allows the team facilitators to identify key areas of success and challenge 
and therefore focus the team’s discussion in areas warranting the most change and improvement. 


District Role in Step 6 


The district-level team also should engage in periodic reflection. Similar to school-level teams, district- 
level teams should consider what is working and not working related to the goals and the tasks 
conducted by the district-level team and examine what the district could do to better support school- 
level efforts through leadership, district-level policies, and access to data and resources (e.g., staff time, 
transportation, support programs, and additional funds). It is suggested that a district-level 
representative either attend school-level reflection meetings or meet individually with a member of 
each school-level team to hear how the team is doing and 
how the district can help. 


Guiding Questions for Step 6 Notes from the Field 
The guiding questions for Step 6 encourage the team to After a year of using EWIS we are 
reflect on each of the five previous steps. integrating it info our work in 
several ways. Our Superintendent 
principal meetings — dedicating a 
* What were the team’s biggest successes? portion of the meeting to using 
EWIS and doing a follow-up cycle. 
* What were the biggest challenges that the team Building-based teams now include 
faced? EWIS in leadership or student 
support team meetings, and next 
* What made the team’s job easier? What changes year we will have a multi-tiered 
could make the team’s job even easier? team look at data together to come 
up with interventions, track studen 
* Who will continue to be on the team? (Note: It is progress, and adjust interventions 


recommended that some individuals overlap to improve outcomes. 
from year to year to ensure long-term 


continuity.) 


¢« Which roles will need to be filled for the upcoming year? Which other constituencies need to be 
represented on the team? 


¢ What is important for new team members to know before deciding whether to join the team? 
What is important to know and be able to do when serving on the team? 


o What changes should be made in how the team organizes its work and functions as a 
team? How should the work be delegated or divided to ensure the fair distribution of 
responsibilities and tasks? 


o When and how frequently should the team meet? 


o How long should meetings be? 
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o Doall members need to be at all meetings? 


o What changes should be made in how the meetings are structured and run? 
o Howmuch time should members expect to need to work outside meetings? 
o What changes should be made in how the team makes decisions? 


o What changes should be made in how the team communicates and shares information 
within the team, with other teams, and with the district? 


o What resources could support the team? 


o Howcould the team integrate its work more fully with existing teams, systems and 
procedures in the district/school? 


What else should the team change to improve how the team organizes its work and functions? 


What support is needed from the district to support the team? 


Step 2: Review EWIS data. 


Were team members who were responsible for generating EWIS reports able to access the 
information easily? 


Did the team feel proficient in using the reporting options? 


Is there any aspect of using Edwin Analytics and/or EWIS that the team would like to learn more 
about? 


Was the team clear about how to review EWIS data? Were the team members clear on the 
questions they hoped the data would help them answer? 


Did the team members feel they had access to the right kinds and amount of data to answer 
their questions? Too much? Too little? 


What challenges did team members face in trying to manage and understand the data they 
received? What kinds of training, guidance, or other support would the team members need to 
build their assessment literacy skills? 


What additional questions about students would the team like to be able to ask and answer in 
the upcoming year? 


What other changes would the team like to see in the amount, nature, format, frequency, and 
timing of the data to best understand the nature and the scope of student needs in their district, 
school, or cohort? 


What else could the team do to improve this step? 


Step 3: Explore Underlying Causes 


How did the team go about trying to discern the underlying cause for risk? 
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Did the team feel this approach was effective (i.e., did the team feel that it was able to identify 


the most important reasons why students were struggling)? 


How much time (e.g., the number of hours) did it take per student or group of students? How 
could this process go more quickly and smoothly? 


Which sources of data were found to be most helpful in understanding student needs and the 
underlying causes for student risk? What other data would the team like to be able to access in 
the upcoming year? 


Was the team satisfied with decisions about which students would be the focus of the team? 


What else could the team do to improve this step? 


Step 4: Take Action 


What challenges did the team face in trying to conduct the inventory of interventions and 
supports? What gaps does the team feel need to be filled in the upcoming year? 


Did the team feel it was able to effectively match students with appropriate interventions or 
supports? How helpful were the EWIS data? The monitoring indicators data? Additional data? 
What else does the team believe could help with matching students to appropriate 
interventions or supports? 


Was the team able to identify schoolwide areas of need based on early warning data? If not, 
how might the team consider broader schoolwide changes in response to EWIS data in the 
upcoming year? 


What else could the team do to improve this step? 


Step 5: Monitor students and interventions. 


How frequently did the team monitor student progress? Does the team feel it should be 
monitoring more often? Less often? With the same frequency? 


What data did the team rely on most to assess student progress? What additional data does the 
team feel should be included next year? 


Are the monitoring flag thresholds appropriate? Should they be adjusted? 


What other sources of data did the team use to determine the effectiveness of interventions 
and supports in meeting student needs? Are there any data sources that the team wants to 
track more systematically in the upcoming year (e.g., participation or attendance in 
interventions or supports)? 


Were the team members satisfied with the data system they used for the monitoring student 
progress? 


By whom and how were interventions and supports managed for students and groups of 
students? Does the team believe that the school should create any new systems or structures that 
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can help manage student interventions and supports? Could these changes help with the 
monitoring of student progress? 


Was the team able to develop any systems or approaches for monitoring student progress that 
worked particularly well? Increased efficiency? 


Were there any interventions or supports that produced greater success than others? For which 
students? Should these interventions or supports be used more widely in the upcoming year? 


Are there any new interventions or supports that the team feels should be offered in the future? 


Are there any district or schoolwide changes that could or should be made to minimize the need 
for individualized supports? 


How effective was the team in using monitoring to help identify and plan for student needs 
during the summer? 


What else could the team do to improve this step? 
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Tool 22 — Step 6: End of the Year Reflection and Sustainability Planning Exercise 


Directions: Within your school or district team, reflect upon and answer the following questions. 


1. What are the most important changes you have implemented this past year as a result of your 
early warning implementation process and use of early warning data? Have you made any 
changes in how you: (1) review student data, (2) identify or support students, (3) assign 
interventions? What other changes have you made this year? 


2. What changes did you hope to implement as part of your early warning implementation, but 
were unable to this year? What impeded this implementation? 


3. List three to four implementation strategies that you used this year and that you are planning to 
sustain, refine, or scale up. 
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Tool 23 — Step 6: Early Warning Implementation Sustainability Action Plan 


Directions: The table provides an opportunity for your team to organize and plan for early warning implementation next year. List your three 
most important implementation strategies in column 1. Respond to the questions in columns 2 and 3 for each of the listed strategies. 


Coumnd Ci Cot ot 
What is the step or change we want to sustain, What have we done, or will we do, to best sustain this) What potential obstacles or competing 
refine, or scale next year? (See section |, question | change for next year? What do we need to make this | commitments might get in the way of this plan? 


happen (e.g., funding, policy changes, staffing, How will we overcome these obstacles? 
culture shifts?) Who will be responsible for these 


Tool 24 — Step 6: Early Warning Implementation Action Planning Tool 


Directions: The following template is designed to support your early warning implementation efforts for Step 6. Teams can identify the key 
objective or task, and then identify the resources available, actions needed responsible parties, and when the task needs to be completed. 


Key Tasks: 

Plan time and approach for reflecting on student progress/outcomes and the implementation process. 
* Gather feedback from team members (e.g., distribute reflection questions). 

« Review team feedback and identify priority areas for discussion. 


Identify areas that are working and explore areas that are not working and how the process could be improved. 
* Document any changes and articulate a plan for implementation. 


Confirm new and continuing team members for the upcoming year and make plans for orienting new members. 
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Section 3. Additional Resources 


For additional information about early warning systems, dropout prevention, and data-driven improvement, please see the following resources. 


Massachusetts Resources 


= Tools: 


¢ Massachusetts District Data Team Toolkit (http://www.doe.mass.edu/accountability/toolkit/): This toolkit is designed to help a district 
establish, grow, and maintain a culture of inquiry and data use that can inform decisions that impact teaching and learning, and 
ultimately improve the achievement of all students. 


¢ Learning Walkthrough (http://www.doe.mass.edu/educators/title-iia/ImplementationGuide2016.pdf): Learning walkthroughs are a 
systematic and coordinated method of gathering data to inform district- and school-level decision making. They involve establishing a 
Focus of Inquiry, and then engaging strategically selected teams of individuals in collaborative observations of classrooms with an 
emphasis on the interactions among teachers, students, and academic content (the instructional core). Learning walkthroughs can be a 
powerful means of helping educators learn more about the ways in which instructional practices support student learning and 
achievement. Evidence from learning walkthroughs can inform analyses of other data, such as EWIS data. 


¢ — District Standards and Indicators Self-Assessment (http://www.doe.mass.edu/sfss/self-assessment.html): The District Standards and 
Indicators self-assessment tool is meant to be used by district teams to support an in-depth collaborative process to measure, track, and 
enhance the effectiveness of the current district systems in a way that is focused on improving student achievement and consistent with 
the state’s expectations for accountability and assistance. 


* Conditions for School Effectiveness Self-Assessment (http://www.doe.mass.edu/accountability/): The Conditions for School Effectiveness 
self-assessment is designed to be used at the school level by leaders, those responsible for day-to-day instruction, and key stakeholders. 
It is a tool for conducting a scan of current practice, identifying areas of strength, and highlighting areas requiring greater focus. 


¢ The Behavioral Health and Public Schools Self-Assessment Tool (hittp://bhps321.org/): This tool is designed for schools to assess current 
activities and strategies that the staff and programs engage in to create a supportive school environment. This tool is intended to assist 
with documenting current practices that support students’ behavioral health at all intervention levels, ranging from the whole school 
community to individual students who require more intensive supports. It also examines the role of various school professionals and 


staff in providing these supports. 
a 
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¢ Massachusetts Guide for Implementing Individual Learning Plans (ILPs) 
(http://www.doe.mass.edu/ccr/schoolcounseling/ilpguidance2014.pdf): This guide was designed for Massachusetts schools and districts 
to support the use of Individual Learning Plans (ILPs) for all students starting in middle school. The ILP is a student-directed, multi-year, 
dynamic tool that maps academic plans, personal/social growth, and career development activities while taking into account the 
student’s unique, self-defined interests, needs, and goals for the attainment of postsecondary success. The ILP implementation may be a 
helpful to support students that are off-track for meeting an upcoming academic milestone. (additional tools at: 


http://www.doe.mass.edu/ccr/schoolcounseling/ ) 


= Information: 
¢ The Edwin website (http://www.doe.mass.edu/edwin/analytics/) includes information about all of Edwin Analytics EWIS tutorials. 


* The dropout reduction website (http://www.doe.mass.edu/dropout/) includes resources and information related to a variety of dropout 
prevention, intervention, and recovery topics and promising practices. An additional tool to look at dropouts can be found 
http://www.doe.mass.edu/ccr/ewi/AnalysisTool.html 


* The Massachusetts Tiered System of Support (http://www.doe.mass.edu/sfss/)is a blueprint for school improvement that focuses on 
system structures and supports across a district, a school, and classrooms to meet the academic and nonacademic needs of all students. 
It was developed to help guide the establishment of a system that provides high-quality core educational experiences in a safe and 
supportive learning environment for all students and targeted interventions and supports for students who experience academic and/or 
behavioral difficulties and students who have already demonstrated mastery of the concept and skills being taught. 


¢ The Behavioral Health and Public Schools Framework (http://bhps321.org/bhps framework.doc) provides a structure to assist schools in 
creating safe, supportive environments and to increase support for students—particularly students with behavioral health challenges— 
through the provision of collaborative services. 


Other Resources 


= Research and information: 


* Developing Early Warning Systems to Identify Potential High School Dropouts (http://www.earlywarningsystems.or 
content/uploads/documents/IssueBrief_EarlyWarningSystemsGuide.pdf): This resource discusses the factors that help predict the 
probability that individual students will eventually drop out of high school before graduating and includes step-by-step instructions for 


building an early warning system. 
—S———_———————————————K$€L$<$<— 
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Approaches to Dropout Prevention: Heeding Early Warning Signs With Appropriate Interventions 
(http://www.air.org/sites/default/files/downloads/report/NHSC_ApproachestoDropoutPrevention O.pdf): This report outlines steps 
that schools can take to identify students in need and provide the necessary support systems and relevant interventions to assist 
students in obtaining a high school diploma. Furthermore, the report discusses the use of early warning data systems to target 
interventions for groups and individual students, offers a variety of best practice approaches undertaken by higher performing high 
schools, and presents effective programs that are currently being implemented to stem the dropout problem. 
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Section 4. Appendices 


Appendix A. Overview: Massachusetts EWIS (updated November 2017) 


Introduction 


The Massachusetts Early Warning Indicator System (EWIS) helps educators understand which of their 
students are at academic risk, and provides an entry point for the use of data in responding to student 
risk. EWIS is based on several years of research examining state data and identifying indicators that can 
most accurately predict the likelihood that a student will either achieve or miss important academic 
milestones. With early identification of students who are at risk, the EWIS allows educators to 
intervene and support struggling students to help them get back on track. 


Background 


The Massachusetts Department of Elementary and Secondary Education (ESE) created EWIS in response 

to district interest in the Early Warning Indicator Index which identified rising grade 9 students at risk of 

not graduating from high school. Districts were interested in understanding students who may be at risk 
in earlier grades and throughout high school. EWIS was created to identify students in grades 1-12 who 

are at risk of missing important and meaningful academic outcomes. In 2016, ESE added postsecondary 

milestones to EWIS to help educators understand whether high school students are on track to succeed 

in postsecondary education. 


ESE collaborated with American Institutes for Research (AIR) to develop the initial EWIS statistical risk 
models for each grade level and age groups (e.g., early elementary school, late elementary school, 
middle school, and high school). Researchers used historical longitudinal data and rigorous statistical 
methods to create a valid EWIS model for each grade level from elementary through high school. 
Although there are some common indicators (e.g. attendance, suspension) across age groups and grade 
levels, the risk models vary by grade level. A team from ESE worked closely with AIR to determine the 
best models for each grade level, and an EWIS advisory group from ESE and other state agencies 
reviewed the research findings and discussed key decisions. ESE has continued to update the EWIS 
models annually. 


Academic Milestones and Age Groups 


For each grade level, academic milestones have been identified that are developmentally appropriate, 
available in state data, important to the success of Massachusetts students, and meaningful and 
actionable for adult educators working with students in each grade grouping. The intent is that 
milestones are scaffolded, with each outcome built on the success in the previous one. 


Exhibit A-1 shows the age groups and the academic outcomes. 
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Exhibit A-1. Massachusetts EWIS Age Groups, Grade Levels, and Academic Milestones 


Early 1-3 Reading by the end of grade 3 
Elementary Meeting or exceeding expectations on the grade 3 ELA MCAS 
Late Middle school ready 
Elementary Meeting or exceeding expectations on the grade 6 ELA and 
mathematics MCAS 
Middle 7-9 High school ready 
Grades Passing grades on all grade 9 courses 


Academic Readiness 
Enrolling in credit-bearing courses without developmental education 


College Persistence 
Enrolling in a second year of postsecondary education 


Postsecondary 


High school | 10-12 High school graduation: 
Completing high school graduation requirements in four years 
College Enrollment 
Enrolling in postsecondary education 


EWIS Risk Levels 

Each student is assigned a risk level for each outcome: High, Moderate or Low risk. These risk levels 
reflect a student’s likelihood of missing an upcoming academic milestone, if no interventions are 
provided. For example, a student with low risk is likely to meet the milestone; a student with high risk is 
likely to miss the milestone. The risk level indicates whether a student is currently off track to reach the 
upcoming academic milestone. 


The risk levels are calculated using data from the previous year and designed to inform supports for the 
next school year. A risk level is assigned to every student enrolled in a Massachusetts public school the 
prior year. Risk Levels are determined on an individual student basis, and take into account the 
cumulative impact of information about the student. There is not a set number of students for each 
risk level. For example, it is possible to have all students in the low-risk category. 


Students enrolled in Massachusetts public schools the prior year, but without sufficient state data, and 
students new to Massachusetts public schools are given a risk level of Not Available (NA). 
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Exhibit A-2. Massachusetts EWIS Student Risk Levels 


Based on data from last school year, the student is ... 
At risk for not reaching the upcoming academic milestone 
High Risk Approximately 25% of high risk students meet this milestone 
Moderately at risk for not reaching the upcoming academic milestone 
Moderate Risk Approximately 60% of moderate risk students meet this milestone 
Likely to reach the upcoming academic milestone 
Low Risk Approximately 90% of low risk students meet this milestone 


Students in grades 10, 11 and 12 may have four risk levels, one for each academic milestone: high 
school graduation, college enrollment, academic readiness and college persistence. For example, an 
11" grade student may be at low risk of missing high school graduation; moderate risk of missing 
college enrollment; and high risk of missing academic readiness and college persistence. This means 
the student is likely to graduate from high school; and if additional support is not provided, is at 
some risk of not enrolling in college, and is unlikely to enroll in credit bearing coursework and a 
second year of college. 


Early warning systems cannot predict with 100 percent accuracy whether students will achieve 
academic outcomes. There will be false positives (students identified as at high risk who end 
up meeting the academic milestone, regardless of additional supports) and false negatives 
(students identified as low risk who fail to meet the upcoming milestones). 


Data used to Create EWIS Risk Levels 


ESE designed EWIS to rely solely on data available from existing statewide collections. The data come 
from several data sources: the Student Information Management System (SIMS), Student Course 
Schedule (SCS) and School Safety Discipline Report (SSDR) data collections, and assessment data from 
State Assessments and the Assessing Comprehension and Communication in English State-to-State for 
English Language Learners (ACCESS for ELLs). EWIS is validated and updated annually as more recent 
years of data becomes available and to account for changes in data sources. 


Exhibits A-4 shows the indicators used for EWIS by grade levels. 
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Limitations and Uses for EWIS 


EWIS was created to help educators understand which students in their schools are most in need of 
additional interventions to reach an upcoming academic milestone. With EWIS and a data inquiry cycle, 
educators can take action to improve students’ experiences. School impacts such as targeted student 
interventions can move a student from being off track, or high risk, to meeting an academic milestone. 
Research is clear that a variety of factors contribute to a student’s likelihood of achieving key academic 
outcomes. 


EWIS is a starting point for identifying and supporting students at risk for not reaching an upcoming 
academic milestone. The EWIS risk level should never be the sole piece of data used for student planning; 
schools should use their local data and additional context — in conjunction with the EWIS Guide -to 
identify, diagnose, and support students. For more on the Early Warning Implementation Cycle, see the 
on-demand e-learning tutorial on the EWIS website. 


For more detailed information on EWIS development, refer to ESE’s EWIS Risk Model Development 
reports, also on the EWIS website. 
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Exhibit A-4. Overview of Indicators used in EWIS Modeling, by Grade Level and Milestones 


Retained? 


Suspensions (in/out of school) peace Xx 
| x | 


Ever retained 


School move? 


Special education level of need 


Gender 


Urban residence 


<x |x |< |x | x< 


Overage for grade? 


CVTE oe 
Schoolwide Title | Xx Xx X 
Targeted Title | Xx fF fT 
ACCESS for ELLs level* x 


ELA State Assessment 
Mathematics State Assessment 


Science State Assessment 


Mathematics course performance® 


Algebra Il or higher® 


ELA course performance? 


Science course performance® 


Social studies course performance? 


Noncore course performance® 


X 
|| 
|| 

Ever ELL | | 
4 
et) 
(a 
|| 
iin 
[ni 
|| 
ea 
Lo 
et) 


Foreign Language 


Pass all courses’ re ee ee 
On Track to MassCore® [Es rs (eel 
Higher Coursework” a ee ee ee 
Ap! a ee ee 
SAT! a ER a Ee Ee 
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Exhibit A-4. Overview of Indicators used in EWIS Modeling, by Grade Level and Milestones Footnotes 

1.Retained students are automatically assigned as high risk in the High School model because 
they are unlikely to graduate on-time. Retained is a student identified as the same grade the 
prior fall as the fall before. 

2.School move captures if a student attended more than one school during the prior school year. 
It does not capture mobility between school years. 

3.Overage for Early Elementary, Late Elementary and Middle School models is defined as one year 
older than the expected age for each grade level. For High School & Postsecondary models, 
overage is defined as students two or more years older than expected grade level. 

4.ACCESS for ELL Levels was introduced in 2012-13; prior to that year MEPA was used. 

5.In the high school models, ELA, Mathematics, Science, Social Studies and Non Core Course 
performance in the High School refer to whether a passed all courses in that subject area, failed 
any courses in the subject, did not complete a course in that subject area (or in some instances, 
No Performance Data Available). 

In the Postsecondary model, in addition to the course performance noted above, the extent of 
course passing is captured with additional indicators, ELA Passing, Mathematics Passing, Social 
Studies Passing and Science Passing. These describe whether a student earned a B or higher for a 
year or more of a course; C or higher for a year or more of a course; Passed a year or more of a 
course; and Pass for less than a year of a course. 

6.Algebra II or higher is defined as taking Algebra II, the equivalent of Algebra II or a course more 
advanced than Algebra II. 

7.Pass all courses means that a student has passed all completed courses (core, non-core and others). 

8. Higher Coursework is defined as students taking Advanced Placement, !B or Dual Enrollment. 

9.On Track to MassCore reflects students’ course-taking progress along the Massachusetts 
recommended program of study. We examine full-year equivalents (FYE) of courses that as 
student has passed, reflected in the Student Course Schedule data available. For more information 
on MassCore, see our assCore website. 


Expected FYE of courses: ELA Math Sci Ss For Arts 
Lang 
Rising 10" grader 
1 1 

(or 1 year of course data) 

woe th 
Rising 11™ grader 3 3 1 1 
(or 2 years of course data) 

woe th 
Rising 12™ grader 3 3 3 5 1 


(or 3 years of course data) 
10. AP refers to students taking the AP test. 
11. SAT refers to students taking the SAT by the end of 11" grade; this field is either Not Taken, Taken 
— College Ready, or Taken — Not College Ready (using the college and career ready benchmark, 
480 in ERW and 530 in math). 
= Additional note: Low-income, as defined by eligible for free and reduced price lunch, was included 
in iterations of EWIS through 2015.In 2015, several communities in Massachusetts participated in 
the USDA Community Eligibility Program and no longer reported students’ free and reduced priced 
lunch status to the state. Without free and reduced priced lunch data from some of the state’s 
largest districts, ESE developed a new metric, Economically Disadvantaged. based on students’ 
participation in other state-administered programs. This measure is not available for previous 
cohorts of students and is substantially different from Low-income. Once sufficient years of 
Economically Disadvantaged data are available, they will be tested for inclusion in the EWIS model. 
For more info on Economically Disadvatged: http://www.doe.mass.edu/infoservices/data/ed.html 
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Appendix B. Accessing EWIS Data and Reports in Edwin (updated November 2017) 


Edwin Analytics is the comprehensive Massachusetts teaching and learning platform that is designed to 
provide state educators with integrated information and tools. Using Edwin Analytics, classroom 
teachers, principals, professional support personnel, and district leaders will be able to access a wide 
range of data reflecting attendance; course enrollment; students’ growing acquisition of new 
knowledge, skills, and abilities; and college matriculation. EWIS is included in Edwin and can be accessed 


from the ESE security portal at https://gateway.edu.state.ma.us. 
Access and Supports 


EWIS offers Massachusetts public school districts information about student risk levels based on the 
prior year’s data. This information is typically available to districts in late August for the start of each 
school year. 


EWIS Access. Districts can access information for any school or student in that district through Edwin 
Analytics. The security portal also can be found via link on the upper-right side of the ESE website 
(http://www.doe.mass.edu/). If you already have a user ID, the next step is to request access to Edwin 
Analytics from your district’s directory administrator. Please see the Access and Security document on 
the Education Data Warehouse public website (http://www.doe.mass.edu/edwin/analytics/) for an 
explanation of these roles. For additional support on accessing Edwin and/or EWIS, please contact your 


district’s directory administrator: http://www.doe.mass.edu/infoservices/data/diradmin/list.aspx 


EWIS Supports. To facilitate districts’ use of EWIS, numerous supports are available, including the 
following: 
= EWIS frequently asked questions and other resources (http://www.doe.mass.edu/ccr/ewi) 


= EWIS video tutorials (http: 
Using EWIS reports in Edwin (http://eoe.state.ma.us/data-in-action/edwin/edwin_ewis%20- 


%20Ed.CR.1/Ed.CR. 1runtime/index.aspx ) 
= Videos on using college readiness data tools (http://www.doe.mass.edu/ccr/resources/data.html ) 


www.doe.mass.edu/edwin/analytics/ewis.html?section=tutorials) and 


Accessing and Maintaining Up-to-Date EWIS Data 


EWIS reports will be updated after the October SIMS is finalized each year. Prior to the October SIMS, 
students are listed in their anticipated grade level. After the October SIMS is finalized, the grade levels 
are updated to reflect any necessary changes (e.g., for students retained in a grade). It is important to 
note that districts can claim any new students to the district (or students attending a new school within 
the same district) at any point in time. By claiming new students frequently, districts can ensure that the 
EWIS reports adequately reflect all currently enrolled students. Districts are strongly encouraged to 
claim students during the summer so that the full list of current students is available to districts when 
the new EWIS data are available at the end of August. EWIS data also will be updated with corrected 
grade levels and school assignments when the October SIMS submission is finalized in late fall. 
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Generating EWIS Reports in Edwin Analytics 


Within Edwin Analytics, districts and schools can choose to generate a range of EWIS reports or export 
EWIS student early warning data files to Excel and conduct analyses and generate reports on their own. 
The EWIS data and reports in Edwin provide districts and schools with several types of reporting options: 


= District View (EW301) and Postsecondary District View (EW303) 

= Graphical View (EW302) 

= Risk Level Indicator Analysis (EW317) and Postsecondary Risk Level Indicator Analysis (EW319) 
= Subgroup Analysis (EW318) 


= Student List (EW601) and Postsecondary Student List (EW602) 


Within each option, districts and schools can filter the data in numerous ways (e.g., selecting specific 
schools, grade levels, and/or student subgroups) to gain varying perspectives on their data and help 
understand the scope, depth, and nature of student risk levels. 


EWIS Reports Located on the District & School Tabs (Public) 


EW301 Early Warning Indicator System (EWIS) District View 

EW301 provides aggregate EWIS student risk levels by school and grade levels. The state view of this 
report shows the breakdowns of student risk levels by grades. The student population in the report 
includes students currently claimed by the District/School and assigned a risk level for the selected 
school year. When viewing past school years, the students’ school, grade, and risk level are based on the 
end of year information for the selected school year. With the appropriate security permissions, users 
can drill-down to the EW601 report to view the student-level information. 


EW303 Early Warning Indicator System (EWIS) Postsecondary District View 

EW303 provides aggregate EWIS student risk levels by school and grade levels for students in grade 10, 
11, and 12 for up to four academic outcomes (high school graduation, college enrollment, academic 
readiness, and college persistence). You may include a bar chart that displays district level results by 
grade for the outcomes selected. The State view displays the results by grade while the District view 
displays the results by school and grade. The student population in the report includes students 
currently claimed by the District/School and assigned a risk level for the selected school year. With the 
appropriate security permissions, users can drill-down to the EW602 report to view the student-level 
information. 


EW302 Early Warning Indicator System (EWIS) Graphical View 

EW302 provides three graphical representations of the student risk levels of not achieving their next 
educational outcome as determined by the Early Warning Indicator System (EWIS) within a district. The 
first is a bar-chart of the risk levels for selected students currently claimed by the district/school and 
who were assigned a risk level for the selected school year. The selected students’ risk levels are 
compared to the district and state risk levels. 


The second chart is a line graph (Section B) and depicts prior student risk levels for the selected 
students. This graph shows the longitudinal risk levels for the selected student cohort. For example, if 5" 
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grade male students selected, the graph shows the most recent student risk level breakdowns for these 
students and the risk levels for these same students when they were in 4" grade, 3 grade, etc. 


The third chart is also a line graph (Section C) that depicts prior student risk levels for the selected 
students. Unlike the graph in Section B, this graph shows the longitudinal risk of the selected student 
population profile. For example, if 5° grade male students were selected, the graph shows the risk levels 
of 5" grade male students this year, last year, two years ago, three years ago, and four years ago 
(presumably most of these students will be different each year). 


EW317 Early Warning Indicator System (EWIS) Risk Level Indicator Analysis 

EW317 is a tool to analyze the patterns of attendance, suspension, state assessment, and course 
performance data that are used to determine EWIS student risk levels. This report is run for a particular 
grade level. The report details, in a table as well as graphically, the indicator information for students in 
each risk level. Only indicators related to the selected grade level in the EWIS model are displayed. 


EW319 Early Warning Indicator System (EWIS) Postsecondary Risk Level Indicator Analysis 

EW319 is a to analyze specific attendance, behaviors, and course-taking or academic trends for students 
in each risk level for a selected grade (10th, 11th, or 12th) and postsecondary outcome (college 
enrollment, academic readiness, or college persistence). This report is run for a particular grade level 
and academic outcome. The report details, in a table as well as graphically, the indicator information for 
students in each risk level. With the appropriate security permissions, users can drill-down to the 
EW602 report to view the student-level information. 


EWIS Reports Located on the District & School Tabs (Unsuppressed) 


EW318 EWIS Subgroup Analysis 

EW318 compares, in aggregate, the EWIS student risk levels across student subgroups (e.g., 
race/ethnicity, high need subgroups. The report includes also a state comparison for the student 
subgroup populations. This data in this report is not suppressed and should be shared with caution. 


EWIS Report Located on the Students, Staff & Classroom Tab 


EW601 Early Warning Indicator System (EWIS) Student List 

EW601 provides a list of all students currently claimed in the district. The list includes the EWIS student 
risk level for each student, along with the indicators used to determine the risk level. Additional 
information about the student that was not used to determine their student risk level (e.g., their prior 
year’s risk level, their race/ethnicity) is also provided. When viewing past school years, the students’ 
school, grade, risk level, and risk indicators are based on the end of year information for the selected 
school year. Due to the student level information in this report the information should be shared with 
caution. 


EW602 Early Warning Indicator System (EWIS) Postsecondary Student List 

EW602 provides a list of the basic student information along with the risk indicators used in the ESE 
Early Warning Indicator System (EWIS) model and the resulting risk level for current students for the 
selected school year. The report includes students currently claimed by the district/school and assigned 
a risk level for the selected school year. When viewing past school years, the students’ school, grade, 
risk level, and risk indicators are based on the end of year information for the selected school year. 
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Box B-1. Student Characteristics: Definition of Terms 


ELL indicates students who do not speak English or whose native language is not English and 
who are not currently able to perform ordinary classroom work in English. 


FELL indicates students identified as not currently ELLs but were ELLs within the past two years. 


Economically Disadvantaged (EconDisadv) indicates a student participating in state administered 
assistance programs 


SWD indicates students with disabilities (i.e., an individualized education program). 


High need indicates students identified as ELL, FELL, Economically Disadvantaged, or SWD. 


Overage for students entering grade 1-9 is defined as one year older than the expected age for 
each grade level (e.g. a student who is 8 or older as of September 1 of their second grade year is 
overage) for students entering grade 10-12, overage is two years older than expected age for 
the grade level (e.g. a student who is 17 or older as of September 1 of their tenth grade year is 


overage) 
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Exhibit B-1. Sample Prompt Page to Select Filtering Options 


District/ Schools 


District 


* [Malden (01650000) =] 


African American/Black 

Asian 

[— Hispanic or Latino 

[ Muttirace, non-Hispanic or Latino 
[7 American Indian or Alaskan Native 
[Native Hawaiian or Pacific Islander 
[ White 


Select all Deselect all 


*| Beebe 


[7 Ferryway 

[- Forestdale 

I Linden 

[7 Malden Early Learning Center 
[TF Malden High 

[- Salemwood 


Select all Deselect all 


Gender 


High Needs 


Low Income 
ELL 
FELL 


Grade Level 


4 


Select all Deselect all 


[- All Students with Disabilities 

Not a Special Education Student 

[— Exited Special Education in Current School Year 

[— Full inclusion (>79%) 

I Partial inclusion (40-79%) 

[~ Substantially Separate (<40%) 

[- Public Separate Day School 

[— Private Separate Day School nal 


Select all Deselect all 
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Exhibit B-2. Sample District View Report (EW301) 


ed EW301 Early Warning Indicator System (EWIS) District View 


© keep thes version + 


> 


& & B- 


Sample District 


EWIS as of Jan-2014 for the 2013-2014 School Year for students currently enrolled in the 2013-2014 School \ ca: 


Schools}: All Schools within Oretect 


Total for Report 
School: x 
Gr 1 
Grade 02 
Grade 03 
Grade 04 
Grade 05 


Grade 01 
Grade 02 
Grade 03 
Grage 04 
Grade 05 


Grade 01 
Grage 02 
Grade 03 
Grace 04 


Grace 05 


Pte wate Rant 


eo 


5 


»>oodoeoecsceeseosoceocoeoesoescoeosowsesese-= 


> Badd the report + 


0.0 
0.0 
00 
00 
00 
00 
00 
0.0 
00 
0.0 
00 
00 
00 
0.0 
00 
00 
00 
00 
00 


an 


122 


Exhibit B-3. Sample Postsecondary District View (EW303) 
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Exhibit B-4 Sample Graphical View Report (EW302) 


© eee tex? - = @ me @* Waster 


Schvvips £W202 Early Warning Indicator System (EWIS) Graphical View 
5" *  EWIS as of Jan 2014 for students currently enrolied in the 2013-2014 School Year 


* Sample District 


x 
choot ” 
wade Lavet 
Section A fiat Lewel for Current Smatonte 
Shckted Chagh Bitgh @ Motes SMedeme Lew 
Seected Staterts Pad , 3 % $2 ma 
Oetm: rr LP] ts. 2 20 1m 
Bae mee 420 «1 aarr st s9 
Section B: Longitudinal Risk of Selected Student Cohort 
100% Risk Level 100% 
@ High 
A > Matermte 


Section C: Longitudinal Risk of Selected Student Population Profite 


a 
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Exhibit B-5. Sample Risk Level Indicator Analysis Report (EW317) 


Beeotermme | b | & Rb- Se Bae 
GCwviri EWS Risk Level indicator Analysis , Sample District 
x 
Grade Lever; & 
Schools Grade Level 
ko P| ‘es = 
p © oJ i 
rx 
iY z 
Tota ror 


Al, Attendance 


0 Gator tond Stusterts 


Ay Mewdarce Rate (%) 
BS Dterts wh Aterdtene ON 
BS Daterts wm Atenas OS 
DS Duderts wth Atendance 150% 

% Daderts wh Atendace « ON 


Var Adendarce Rute (% 
Mas Attendee Rate (%) 


S Duderts eth Atendance > 71% a“ nm " 
WS Ttuderts oath Amendance 91% ar Ph) “4 
S Students ath Attendance 0 OS at] 


% Students wth Aterdance « 00% ‘ 


A2. Suspensions 
Mp Une ee lo 


© Selected Studerts a“ “) 


‘wagertane 
oA . BS eters we 0 Kapereors 


BS Deets we | Sapereen 
OS Duterts wm 2+ Cnpersers 


Wr napenkans 0) 
Mae Setgpersare (6) ’ 


% Ruderds et 0 Suspermons 4 1} x 


Ss Studerds eft | Sutpetect sP ? % yp thode ae Low 


RS ts att Pe Crees 
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BT. MCAS ELA 


6 Selected Shaderts 


% Adeances 

% Profcvert 

S Needs lmorenert 
% Waneg 

% Miesning MCAS 


82. MCAS Math 


@ Selected Suuderts 


Advanced 

Ss Probert 

Se Newes ingrnement 
N Wianey 

% Mevsing MCAS 


C1, Passing Course by Subject 


0 Selected Stasects 


ts Pons BLA 

%\ Pans Math 

% Pass Sconce 

% Pans Socal Studer 
% Pass Nien Core 


C2. Course Performance Overview 


© Selected Stterts 


Ag Course Subjects F ates (#) 


‘% Fated No Course Subjects 


% Fasled 1 Course Subyect 
%s Fnied 2 Course Subjects 
‘% Fated ) Course Sutpects 
S Faled 4 Course Subects 


” 


Hap 


“a 


sop 
“9 


Medeute 


60 


100 


Section B: MCAS - ELA & Math 


BBs Aseces 

Ov toes 

[Bs reeds imgrovement 
Bs vier 

EE % essing cas 


BBs soccer 

Bs test 

1) § tices ingrrenert 
Bs vor, 

GB) 8 Maseg ucas 


| Pee) 

GX Pans taen 

GBS Pers Sconce 
GBS Pots Sects Stutes 
GBS Pass non-core 


BBS fetes No Course Streets 
Bs Fates | Course S.tyect 
GBS Fates 2 Courne Suspects 
GBS fates 3 Corte Setpects 
GBS Footed 4 Course Sutpects 


GS fates 5 Course Suspects 
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Exhibit B-6. Sample Postsecondary Risk Level Indicator Analysis Report (EW319) 


edvvin 


Postsecondary Risk Level Indicator Analysis 


Sample School 
School: me 


Grade Level: 11 
Outcome: College Enroliment 
Grade Level 

11 ’ 
Schools 


“ Sample School 


Outcomes 

* College Encolimernt v 
Sections 
"| AF Secton 5. Attendance 

F¥ Sector 2 Betenor 

7 Secson 3 Course Pertormane 

WF Sector 4: Higher Academies 

fF Secton 5 Assessments (MCAS and SAT) 


Soiesias Qeacies: 23 


Report Overview 


@ Selected Stucerts os 


Section 1: Attendance 
Low 
8 Selected Students 12 7 28 


Studerts wih Attendance < 50% 
Students wan Attenderce 6.90% 
Students wih Atiendance 90-95% 
Students wth Attendance » 55% 


BE 
AB ie 
i in t= 


& & 
@ 
t3 


a 
z 


i 


ad 5% rls 
aa 
Sample District 


School Yeas: 2017-18 


Bh % Studerts wth Aterctance » 05% 
BS Studerts wen Amerdance 90.95% 
(i % Suderts wth Aterdance 00.90% 

© Suderts wih Atersiance « 80% 


Low 


OB % Studeres wih 0 Suspersens 
BB % Stucerts wih 5 Sutpenon 
OD % Studeres wih 2+ Suspensions 


Low 
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Section 3: Course Performance 
Section 3a: Course Passing 


High Moderete . Low 


8 Selected Stacerts t= fe) 238 
%S panning CLA ts % 0 
*s passes Mecr 4e ee | 100 
%S pasaing Science “o 7 0 
% passing Social Stuctes ae ~ 
S passing Foreign Language 8 & * 


Section Dh: ELA Course Performance 


Hge Modensts 
’ “s 8 % 
& Selected Sucerts 1x 
Pass ee 3 # 
Fos saiu 3 ‘ 
Ne Course Completed 5 0 


8 Selected Shderte '% Me 
Pose 26 RR @ 
Fat ais ¢ 
to Cowse Completed i 0 0 


Section 34: Science Course Performance 


8 Selecies Sauderts 1s 78 
Pose yuwn 
Fat gs» 6 
No Coune Compieand avy ' 


Low 


Section 3e: Social Studies Course Performance 


gh =—- Mocerste 
ei *is & 


8 Selecies Stuterts 180 mB 
Pose @«m # 
Fe ms) 4 
Neo Course Completed x 2 0 


low 


f 


Np Conrpe 


low 
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Section ©: Foreign Language Course Performance 
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—_Z Moser aie Low 
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8 ete te) ON aterts % P : ; 

7 
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Section 4: Higher Acedemics 


Section 4; Higher Coursework (AP, IB or Dual Enrollment courses) 
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Section 5a; ELA MCAS 


© Semectos Snuderty 50 r 20 
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town e « “ 
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No Cate pendants se: 2 e ¢ 
Section Sh: Math MCAS 
‘ig Moderate Low 


© Setectes Sasderts $ 7 
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Needs imorovernert £3 2 23 8; 4 
eres a3) 19 ! $ 
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Exhibit B-7. Sample Subgroup Analysis Report (EW318) 


> is so Br | a 


{ , rt} erry 
edwin EWIS Subgroup Analysis Sample District 
"=" = Contains personally identifiable information. Please do not distribute. : : 
School Year; 2017.18 


School: Al Students within Distnct 


Grade Level: 01, 02, 03, 04, 05, 06, 07, 08, 09, 10, 11, 12 
Schools Grade Level 


~ Submit 


Data shown for grades: 01, 02, 03, 04, 05, 06, 07, 08.09, 10, 11, 12 


Selected Students in District Selected Students in State 
Total Students High = Moderote Low WA Total Students High Mocerste Low NA 


‘ ’% a ac cee a ee ee 
“9 S352 184416 18 189774 2 OTS $ 1600 0 
20 T6787 227s 3 IW WD ws To OO 
39 7785 626 1) «ee ts ame 7 2 o OI 
39 166,007 S604) 35 $520 33 $3103 AD BB OO 
10 Wi 520 18 738 BM 174 3 OD OO 
0 0 1,607 46 3 40 35 oy oS 406 
0 0 7% ime 7 | 6a 2 8 
50 $19,622 GOGST 12 On 18 368673 71 1,008 0 


415,709 S7HG 1 G71 21 OO OS OTM CO 
80 437660 410 2 i126 23 27S SM 1065 6 


» 
oo 


M2008 12540 MB 4 BM 17S M 12h 6 
339 MET BD BOIS VW HST HB 8 O 
0 
0 


70674 41418 $2 MER NH 14D 7 164 
645 2 S87 33 NAD 43 7 
3657 G64 46 I ON 428 OH Toe +t 
8062 SOM 35 4376 2 See 4 6 
282 HSE OWI COE 9 «(0 
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ooeoeaoogesosd =‘ & 
x 
% 


130 


Exhibit B-8. Sample Student List Report (EW601) 


Note: due to the length of this report, the table sample is represented in two pieces below. 


ech ~~” W601 Early Warning Indicator System (EWIS) Student List 
“WE *  EWIS as of Sep 2017 for the 2017-2018 School Year for students currently enrolled in the 2017.2018 School Year 


Schoolisk = Al Schools within Distrkt 
Grade Level: 08 


Middle School: Passing all grade $ courses 


Shoot Rsk = Attend 44 Scho SPED Urban 
Nome Grade = Lost Name Fest Name SASiO Level Rate suspension Retgined Level GeOdE asin OOH cr rorriige tr Ells 
ots, 


Sample Student 


ELAMCAS Math MCAS 


2017 2017 


Expectations 
Expectations 
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Exhibit B-9. Sample Postsecondary Student List Report (EW602) 


Note: due to the length of this report, the table sample is represented in three pieces below. 


ed) . "s EW602 Early Warning Indicator System (EWIS) Student List--Postsecondary 
LVEED ewis as of Sop 2017 for the 2017.2018 School Year for students currently enrolled in the 2017.2018 School Year 


inh Oo 


Scheels: = Arrescury nige: 
Grade Level: 10, 11. 12 


Postsecondary: Postsecondary enrollment, readmess and persstence 


School Cofege = Academic Atvens Ever 
fisme | O08 Lest teaene First Marne SASID Ervorent Readiness POMMeree “nu, Suspension Retsined os 


V wm 0 Ne Ne 
S P ow Modertin High Nigh eo 
EVAS infoemation 
BA Me xi Mah ap OUGA &i 
Stoo «= SPED Yuet = ACCESS. Ever Marn ELA 
Geeder Oversge OVTE MCAS MCAS MCAS Course ~ Cone com 
Move = Level Scnoclaide Or Ells ELL mS HS HS Part posting Higher Peet passa Ped 
te Modermte Male = th Nw No Nore No NM NM NM Pes Pose Ne Poss Pose Pas 
eu eu 
thar than 


courses ~Masscore Coursework 


No Course Pess All No No No Not 161 ¥ White N WA Parbal 
Cornpleted Taken Inclusion 


Exporting EWIS Data to Excel 


Some districts may want to expand their capacity to examine EWIS data beyond the reporting options 
offered through Edwin Analytics and therefore may choose to export their EWIS data into Excel as a 
separate EWIS data file. Districts can export EWIS data within Edwin Analytics to Excel by using the drop- 
down box on the upper-right side of the EWIS report screen. 


> lcs Br | we | 
SB View in HTML Format 
ta View in POF Format 


3) View in CSV Format 


Or you can choose to run the EW601 and EW602 reports directly as in excel by clicking the logo 


nextto the reportnames (circled below). 


Edwin Analytics 


Home District & School - Public ee Student Es Ecusatee Prep 


Early Warning indk ator 
System (EWS) Student List 
(Evvé0+) 


Early Waring indkator 
System (EWS) Stucent 
List-Postsecondary 
(Ew602) 


Earty Warning Indicators L Classroom =F Postsecondary Readiness & Success” _ ] 


Early oor Indicators 
Aggregate and stedent level reports containing information about early warning indicators for K 12 students. Additional information 


about EWIS is available om the EWS public rerbpare. More information about these reports; [Wil 


Early Warning ings. ator System (EWIS! Stugent Sst (EWE! ef sans 
Details the risk indicators used in the ESE Earky Warning Indicator Sy WIS) model and the resulting risk levels of not 
achieving their next educational outcome for current students. 


Detads ihe risk indicators wal in the ESE Early Warning Tadinator System (EWIS) =e) the resulting risk levels of current 
high school students not achieving their educational outcome im postsecondary education. 


After the EWIS data are exported, districts and schools can combine these data with other sources of 
district and/or school data and/or generate a wider number of reports. 
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Appendix C. Sample Early Warning Implementation Timeline 


August/Early 
September 
(Before start 
of school) 


September- 
October 


October- 
November 


December 


Sample Activities 


Step 1: Get Organized 


Determine team structure, composition, roles, responsibilities, and goals 

Confirm access to EWIS data in Edwin Analytics 

Identify student data sources to be used for monitoring and establish thresholds for 
flagging 

Set data import and team meeting schedule for the year 

Identify agenda items for the first team meeting of the school year 


Step 2: Review EWIS Data 


Convene first team meeting of the school year 

Determine key questions and focus for data exploration 

Generate EWIS reports to answer questions 

Identify patterns in EWIS data and summarize key findings 

Identify individuals or groups at risk, and/or grade/schoolwide areas of concern 


Step 3: Explore Underlying Causes 


Consider possible factors contributing to student risk for identified students 


Determine data collection plan (how/when/who will collect this additional data to 
confirm likely underlying causes) 


Review additional data to discern likely underlying causes of risk 
Confirm problem(s) to be addressed and potential interventions needed 


Create inventory of available interventions and supports 


Review student needs and available interventions and supports to identify 
opportunities and gaps. Check initial attendance data to ensure all at-risk students 
have been identified. 


Confirm priorities for meeting student needs 


Develop a plan for effectively using available resources to address individual, group, 
and schoolwide needs 


Assign, document, and communicate about planned interventions and supports 


Step 5: Monitor Students and Interventions 


O 


a 
O 
O 


Import monitoring data (e.g., course grades, discipline records, attendance) for first 
grading period 


Review monitoring data to identify flagged students 

Revisit Steps 3 and 4 (Explore Underlying Causes and Take Action) for students 
identified as at risk 

Communicate with teachers, specialists, students, and families to assign and/or adjust 
interventions and supports 
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| Month = Sample Activities 


Steps 3- 5: Explore Underlying Causes, Take Action and Monitor Students and 
Interventions 


January- 
March 


April-June 


Summer 


Review monitoring data to identify students with risk flags 

Revisit Steps 3 and 4 (Explore Underlying Causes and Take Action) for students 
identified as at risk 

Communicate with teachers, specialists, students, and families to assign and/or adjust 
interventions and supports 

Monitor progress of interventions and explore broader schoolwide changes to address 
the needs of many students 

Import monitoring data (e.g., course grades, discipline records, and attendance) for 
third grading period 


Steps 3- 5: Explore Underlying Causes, Take Action and Monitor Students and 
Interventions 


O 


O 
O 
O 
O 
O 
O 
O 


Review monitoring data to identify students with risk flags and to guide planning for 
summer interventions and supports 


Revisit Steps 3 and 4 (Explore Underlying Causes and Take Action) for students 
identified as at risk 


Monitor progress of interventions and explore broader schoolwide changes to address 
the needs of many students 


Communicate with teachers, specialists, students, and families to assign and/or adjust 
interventions and supports 


Step 6: Refine EWIS Process and Step 1: Get Organized 


Reflect on team functioning and refine procedures for: importing data, reviewing data, 
and assigning interventions 


Establish team composition, schedule and goals for upcoming year 


Consider broader schoolwide changes to address the needs of many students 
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Appendix D. The Postsecondary EWIS (September 2017) 


Why use a Postsecondary EWIS? 


The Department is committed to preparing all students for success in the world that awaits them after 
high school. Our mission is to strengthen the Commonwealth's public education system so that every 
student is prepared to succeed in postsecondary education, compete in the global economy, and 
understand the rights and responsibilities of American citizens, and in so doing, to close all proficiency 
gaps ° 


Although Massachusetts is an education leader nationally and 70% of public high school graduates 
enroll in postsecondary education immediately after graduation, there are inequities in college 
engagement across the state, specifically among racial & ethnic groups and income levels, students 
who are English learners, and students with disabilities. To support schools and districts in 
understanding whether their high school students are on track to succeed in postsecondary 
opportunities, EWIS includes three college success milestones. 


Postsecondary Milestones 


Massachusetts EWIS, or Early Warning Indicator System, is a tool to systematically identify 
students that may need additional attention in order to reach an upcoming academic milestone. 
To build context and understanding, educators may wish to learn to what extent their former 
students have met postsecondary milestones in prior years. School and district results for 
postsecondary outcomes are on the ESE website in Profiles (see Students — Graduation Rate); in 
the District Analysis and Review Tools (see Success after High School DART) and in Edwin (see 
Postsecondary Readiness & Success reports). Instructional videos on the College and Career 
Readiness Data Resources (http://www.doe.mass.edu/ccr/resources/data.html) can help users 


navigate these resources. 


The Massachusetts EWIS identifies students in grades 1 through 12 as high, medium and low risk 
of missing important academic milestones throughout the academic trajectory, including college 
success. There are three EWIS Risk Levels: Low Risk, Moderate Risk and High Risk. EWIS Risk Levels 
quickly provide information to educators about the likelihood that a student will miss an upcoming 
academic milestone. For example, a student identified as Low Risk is likely to meet the academic 
milestone; a student identified as High Risk is unlikely to meet the academic milestone. Exhibit A-1 
shows the EWIS Academic Milestones. 


5 For more, see the ESE Strategic Plan Summary (http://www.doe.mass.edu/research/StrategicPlan- 
Summary.pdf) and the BESE’s definition of college & career readiness 
(http://www.doe.mass.edu/bese/docs/fy2016/2016-01/joint-item1.html ). 
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Exhibit -A1. Massachusetts EWIS Age Groups, Grade Levels, and Academic Milestones 
[early '1-3. __—|| Reading by the end | Reading by the end of grade 3 grade 3 
Elementary Meeting or exceeding expectations on the grade 3 ELA MCAS 
Late Middle school ready 


Elementary Meeting or exceeding expectations on the grade 6 ELA and 
mathematics MCAS 


Middle 7-9 High school ready 
Grades Passing grades on all grade 9 courses 
High school | 10-12 High school graduation: 
Completing high school graduation requirements in four years 
College Enrollment 
Enrolling in postsecondary education 
Academic Readiness 
Enrolling in credit-bearing courses without developmental education 
College Persistence 
Enrolling in a second year of postsecondary education 


A Risk Level is assigned to every student who was enrolled in a Massachusetts public school in the 


Postsecondary 


EWIS Risk Levels 


prior year (and for whom there are state level data), and for each outcome. The Risk Levels are 
determined on an individual student basis using data from the previous school year, and reflect 
students’ risk at the beginning of the school year. A student in grades 10-12 may have a Risk Level 
for each of four milestones. For example, a student may be Low Risk for graduating high school, 
Moderate Risk for enrolling in college and High Risk for academic readiness and persisting in college, 
depending on the student’s specific indicator data. Exhibit A-2 shows the Risk Levels. 


Exhibit 2. Massachusetts EWIS Student Risk Levels 


At risk for not reaching the upcoming academic milestone 

| Approximately 25% of high risk students meet this milestone 

| | Moderately at risk for not reaching the upcoming academic milestone 
Approximately 60% of moderate risk students meet this milestone 


Likely to reach the upcoming academic milestone 
Approximately 90% of low risk students meet this milestone 


To develop the EWIS Risk Levels, researchers use a rigorous statistical method?? to create a valid EWIS 
model for each grade level and outcome. The postsecondary models include historical longitudinal data 
from ESE data collections and SAT and AP from College Board. National Student Clearinghouse (NSC) 
data are used in developing the college enrollment and college persistence models. Higher Education 
Information Resource System (HEIRS), student data from public higher education institutions across 
Massachusetts, are used in developing the academic persistence models. (Approximately 30% of the 
college-going high school graduates across the state attend Massachusetts public institutions.) Altogether, 
the model allows users to identify students who are at risk of missing key academic milestones within 
the K-12 and postsecondary educational trajectory. 


Postsecondary EWIS Reports 


ESE collaborated with partners using EWIS in the field to develop and test EWIS Postsecondary 
reports. The designs of the reports support their use. With the District/School View Report (EW303) a 
user can get an overview of student risk across the postsecondary milestones, and use this to set 
priorities. The Indicator Report (EW319) is helpful for investigating patterns in indicator data for 
groups of students. The Postsecondary Student List (EW602) is useful for identifying students’ unique 
strengths and needs, and combining with local data for diagnosis and monitoring. The reports can be 
filtered using demographic fields, and can be downloaded for analysis or sharing, consistent with 
district confidentiality guidelines. Several reports are hyperlinked to other reports for additional 
details. Instructional videos on the College and Career Readiness Data Resources 
(http://www.doe.mass.edu/ccr/resources/data.html) can help users navigate these resources. 


Educators can use the reports to understand the broad trends within their school or district and 
(alongside local data) to diagnose students’ needs. Once the underlying causes of risk are understood, 
educators can provide interventions to meet students’ strengths and needs using the Early Warning 
Implementation Cycle. See the Early Warning Implementation Cycle on-demand e-learning tutorial on 


the EWIS website (www.doe.mass.edu/ccr/ewi ) for more. 
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Resources 

Teachers, guidance counselors, school leaders and other school staff — people equipped with 
information about high school graduation and success after high school — are important points of 
contact for students. These adults may help students understand relationships between their interests 
and dreams, and the education and training requirements needed to get there. Talk to your students 
about the importance of attendance, behavior and courses in high school. Hold high expectations for 
students, let them know when they’re doing well, and talk about how to be even more successful next 


time. See these resources to learn more about supporting students in your community. 


e ESE’s Resources 


o MassCore (http://www.doe.mass.edu/ccr/masscore) 
o  EWIS (http://www.doe.mass.edu/ccr/ewi ) 


e The What Works Clearinghouse (WWC) reviews the existing research on 
different programs, products, practices, and policies in education to provide educators with the 
information they need to make evidence-based decisions. We focus on the results from high- 
quality research to answer the question, “What works in education?” 
o College Bound with What Works Clearinghouse 
https://ies.ed.gov/ncee/wwc/collegebound1 
© Helping students navigate the path to college — what high schools can do 
https://ies.ed.gov/ncee/wwc/PracticeGuide/11 


e FutureReady MA is a public campaign to enable communities to promote understanding and the 
use of various tools to advance the college-and-career-readiness of the Commonwealth’s 
students. Read the blog, sign up for the newsletter and examine the tools to help your students 
Start Now, Aim High and Look Beyond. 


o Find more at: http://futurereadyma.org 


e The College Readiness Indicator Systems (CRIS) initiative was developed in response to a 
troubling pattern: More students than ever are enrolling in college after high school, but many of 
them are not college ready, as evidenced by persistently low rates of college completion. The 
sense of urgency to close the gap between college eligibility and college success is a growing 
concern among policymakers, educational leaders, and the business community. 

o The CRIS Framework has three distinct & interdependent college readiness dimensions: 
academic preparedness, academic tenacity, and college knowledge. 

=" Academic preparedness refers to key academic content knowledge and 
cognitive strategies needed to succeed in doing college-level work. Examples of 
indicators of academic preparedness are student GPA and the availability of 
Advanced Placement courses at a school. 

=" Academic tenacity refers to the underlying beliefs and attitudes that drive 
student achievement. Attendance and disciplinary infractions are often used as 

SSS SSS 
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proxies for academic tenacity; other indicators include student self-discipline 
and the extent to which teachers press students for effort and rigor. 

=" College knowledge is the knowledge base and contextual skills that enable 
students to successfully access and navigate college. Examples of college 


knowledge indicators are students’ knowledge of the financial requirements for 
college and high schools’ promotion of a college-going culture. 
o Find at: https://gardnercenter.stanford.edu/publications/college-readiness-indicator- 
system-cris-resource-series 


Implementing Academic Interventions from REL Southeast 
0. This info-graphic illustrates the necessary components in the complex process of 
developing and delivering academic interventions successfully. 
o  https://ies.ed.gov/ncee/edlabs/infographics/pdf/REL_SE Implementing high school aca 
demic_interventions.pdf 


Minimum Admission Standards for State Universities and UMass from Massachusetts 
Department of Higher Education 
oO This Reference Guide provides detailed information about the minimum standards for 
undergraduate admission to Massachusetts’ state universities and University of 
Massachusetts (UMass) campuses (UMass Amherst, UMass Boston, UMass Dartmouth, 
and UMass Lowell). 
o http://www.mass.edu/foradmin/admissions/documents/DHEAdmissionsStandardsRefere 
nceGuide%20June%202016. pdf 


The definition of College & Career Readiness from Massachusetts Board of Elementary and 
Secondary Education 
o Massachusetts students who are college and career ready and prepared for civic life 
demonstrate the knowledge, skills and abilities that are necessary to successfully 
complete entry-level, credit-bearing college courses, participate in certificate or workplace 
training programs, enter economically viable career pathways, and engage as active and 
responsible citizens in our democracy. 
http://www.doe.mass.edu/bese/docs/fy2016/2016-01/joint-item1.html 
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Appendix E. Targeted and Schoolwide Intervention Resources 


Your review of early warning data may suggest the need for targeted supports for students at risk, as well as broader schoolwide supports. 
Adopting targeted interventions can provide support tailored to the specific needs of groups of students at risk. Schoolwide interventions can 
help reduce risk for all students and, therefore, can be a cost-effective and proactive approach to addressing elevated levels of risk among all 
students within your school. The following table lists a number of resources that provide inventories of available interventions designed to 
address a range of student needs, from academic achievement to mental health. It is important to note that this is not a list of recommended 
interventions, nor is it an exhaustive list of available intervention resources. Rather, this list of resources can help you identify suitable 
interventions aligned with your school’s and students’ needs, and assist you in evaluating the interventions that are likely to meet the needs of 
students within your school’s context. 


The targeted and schoolwide intervention resources are listed alphabetically in the intervention matrix. Within the matrix, you will find a brief 
description of each resource as well as some information regarding intervention types, the appropriateness of listed interventions by school 
levels, and the level of research review and rigor of evidence the resource/site required for including interventions. For information on how we 
classified resources’ intervention type, level, and rigor of review, please see Appendix A— Matrix Key. 


Rigor of 
eesoutce Brief Description Intervention Type hess 
Review 


Best Evidence euidenee ‘Searchable-del database of educational programs, grouped by grade level MiAcademic veers 
Encyclopedia and category (including a comprehensive school reform section); Osocial-emotional / MMiddle 
identifies top-rated programs. Behavioral health MHigh 


Users can access reports, educators’ summaries, and educator guides Ospecial populations 


discussing how programs were assessed for evidence of success. 


http://www.bestevidence.or, 


Blueprints for Searchable database of 47 model and promising youth promotion MAcademic MElementary 


Healthy Youth programs, all of which have been reviewed and recommended by an MSocial-emotional / MMiddle 


Development independent panel. Behavioral health MHigh 
Users can filter interventions by target outcomes, target populations of Ospecial populations 
students, program categories, program setting (including school), and 
risk and protective factors. 


http://www.blueprintsprograms.com/programSelector.ph 
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Rigor of 
Review 


— Trends’ 
What Works 


Collaborative for 
Academic, Social, 
and Emotional 


Learning 


College & Career 
Readiness & 


Success Center at 


American Institutes 


for Research 


Early Warning 


Systems in 
Education 


Searchable database of more than 600 out-of-school interventions 
targeting healthy child development, all of which have been reviewed 
by researchers. 


Users can filter their searches by age / grade level, by program setting 
(including a “school-based” option), by type of program, and by 
outcome. 


http://www.childtrends.org/what-works/ 


List of social-emotional learning programs that have been vetted and 
coded by researchers, with a particular focus on elementary-aged 
students. 


Users can access a profile of each intervention, which includes a 
summary of the intervention, the grade range covered, the program 


context (e.g., schoolwide) and evidence of the program’s effectiveness. 


The site in is the process of developing a list of social-emotional 
learning programs targeting middle and high school students, with an 
anticipated release date of spring 2014. 

uide 


http://www.casel.or, rograms 


Searchable database that includes research articles, tools, and 
products, all of which have been screened for the extent of their 
research base. 

Users can search the database using key words. 


http://www.ccrscenter.or 


Searchable library of materials focused on early warning systems as 
effective, research-based interventions for students at risk. 


Users can access a list of approaches to dropout prevention and 
featured examples of effective programs, and also can access tools for 
monitoring early warning system indicators. 


http://www.earlywarningsystems.org/resources- 
tools/implementation-and-research 


MAcademic 
MSocial-emotional / 
Behavioral health 
OSpecial 
populations 


DAcademic 


MSocial-emotional / 
Behavioral health 


OSpecial populations 


MAcademic 


OSocial-emotional / 
Behavioral health 


OSpecial populations 


MAcademic 


OSocial-emotional / 
Behavioral health 


OSpecial populations 


leicleriantay 
viMiddle 
MHigh 


MElementary 
MiMiddle 
MHigh 


MElementary 
MiMiddle 
MHigh 


MElementary 
vIMiddle 
MHigh 
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Resource 


| FindYouthInfo 
Program 
Directory 


National Center 
on Intensive 
Intervention 


National Dropout 
Prevention Center / 


Network 


Brief Description 


Searchable directory of evidence-based programs designed to reduce 
problem behaviors in youth.® 

Users can customize searches by keyword and by main risk and/or 
protective factor; results can be sorted by age range and by rating 
(determined by study quality and program impact). 


http://www.findyouthinfo.gov/program-directory 


Interactive chart listing studies of intensive academic interventions is 
featured; also hosts a list of behavioral and academic progress 
monitoring tools and a library of webinars around intensive 
intervention topics. 


Users can use the chart to examine intervention studies and find 
information about target population risk levels, the quality of the study 
design, the fidelity of implementation, whether study measures were 
targeted and/or broad, the effect size found, the intensity of the 
interventions, and whether additional studies have been conducted on 
the specific intervention. 


http://www. intensiveintervention.org/chart/instructional- 
intervention-tools 


Searchable database of strategies and programs deemed effective for 
dropout prevention, in two formats: a list and a model programs 
database. 


Users can refine searches by grade level, evidence level, program 
emphasis, target groups (including a schoolwide option), risk factors, 
and more. 


http://www.dropoutprevention.org/effective-strategies 


| Rigor of 


DAcademic 


MSocial-emotional / 
Behavioral health 


MElementary 
viMiddle 
MHigh 
OSpecial 

populations 


MAcademic 


OSocial-emotional / 
Behavioral health 


MElementary 
MiMiddle 
OhHigh 
MSpecial populations 


MAcademic 


OSocial-emotional / 
Behavioral health 


MElementary 
MiMiddle 
MHigh 
OSpecial populations 


§ The FindYouthInfo Directory searches two databases simultaneously: the Teen Pregnancy Prevention Program Directory and crimesolutions.gov, the latter of 
which overlaps with the Office of Juvenile Justice and Delinquency Prevention’s Model Programs Guide. 
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National Registry of 


Evidence-based 


Programs and 
Practices 


Office of Juvenile 
Justice and 


Delinquency 
Prevention’s 


Model Programs 
Guide 


Social and 
Emotional 
Learning 
Research: 
Evidence-Based 


Programs 


Searchable registry of more than 310 evidence-based interventions 
that promote mental health and safe behavior. 


Users can customize searches by age group, setting (school), 
outcome categories, and more. 


http://www.nrepp.samhsa.gov/AdvancedSearch.aspx 

Database of research-based youth intervention programs that are 
classified as “effective,” “promising,” or “no effects” by a panel of 
researchers. 

Users can review and search interventions by topic, as well as by 
focus (e.g., dropout prevention, school climate), age range, and 
protective and risk factors. 
http://www.ojidp.gov/mpg/Topic/Details/107 

List of social and emotional learning programs that have shown 


evidence of effectiveness, as supported by at least three 
independent evaluations or publications. 


Users are provided information on program methods and associated 
outcomes, as well as a summary of the evidence base and citations 
for relevant studies. 


http://www.edutopia.org/sel-research-evidence-based-programs 


DAcademic 


MSocial-emotional / 
Behavioral health 


OSpecial populations 


DAcademic 
MSocial-emotional / 
Behavioral health 
OSpecial 
populations 


DAcademic 


MSocial-emotional / 
Behavioral health 


OSpecial 
populations 


MElementary 


MMiddle 
MHigh 


MElementary 
MIMiddle 
MHigh 


MElementary 
MIMiddle 
MHigh 
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Targeted and Schoolwide Interventions Matrix Key 


Table 1. Intervention Type provides a description for each of the types of interventions covered by our listed 
resources. 


Academic Interventions targeting academic achievement, 21st century skills, study skills, and 
vocational / life skills 


Social-emotional / Interventions targeting social and emotional development, mental health, enhanced 
Behavioral health behavioral supports and / or the promotion of healthy behaviors and the reduction of 
risky behaviors (e.g., delinquency, drug use) 


: : Interventions targeting students with higher needs for support, such as students with 
Special populations aoe ae : sete 
disabilities or limited English proficiency 


Table 2. School-Level Classification provides the grade levels for which the listed resource provides interventions. 


Elementary Targeted at students in kindergarten through Grade 5 
[Middle =i Targeted at students in Grades 6 through 8 
[High = sd Targeted at students in Grades 9 through 12 


Table 3. Rigor of Review indicates the extent to which research on the intervention is reviewed and included in the 
resource. For further information on the criteria for inclusion established by a given source, follow the embedded 
link in the Rigor of Review column. 


No Information There is limited or no information about how interventions are identified for inclusion in the 
(NI) list of interventions. In some instances, this may be user driven. 


Research on each intervention is reviewed. Interventions may be included with research- 
Low (L) based evidence of positive impact determined solely by the research authors (i.e., without 
an independent review). 


Research on interventions is reviewed. The quality and rigor of the research are confirmed 

by an independent source before it is included in the assessment of intervention 
Moderate (M) é if : ae 

impact/outcomes. However, the standard for research to be included is varied in terms of 


the size of the study and the rigor of the methods. 


Research on interventions is reviewed and confirmed by an independent source. Only large- 
High (H) scale studies and randomized controlled trials are included as evidence of an intervention’s 
impact. 
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